19 Selected Advanced Topics

The goal of this section is to introduce selected advanced programming techniques in Python. Since Python is a modern language which is still evolving, we recommend that after reading about a technique here, you also search it on the web for possible updates.

19.1 Objectives

You will learn about:

• Maps and filters.
• The built-in function reduce.
• Creating shallow and deep copies.

19.2 Maps

You already know list comprehension well from Subsections 7.29 - 7.32. You also know that comprehension works for any iterable, not only for lists. The difference between a map and a comprehension is rather subtle, and many programmers prefer comprehension for being more Pythonic. But it is a good idea to learn about maps anyway because there are many other people who use them.

The built-in function map(fun, iterable) applies the function fun to each item of the iterable (text string, list, tuple etc.). The function fun can be an anonymous lambda expression or a standard function. For anonymous lambda expressions review Subsections 8.18 - 8.26. The result returned by the map function is a Map Object which needs to be cast back to an iterable.

In the following example, the function fun is given as a lambda expression, and the iterable is a list of numbers [0, 1, ..., 9] created via range(10):

Let’s take a minute to do the same using list comprehension:

You can see that a cast to list is not needed in this case. As another example, let’s show that map can use any function, not only lambda expressions:

And last, let’s show that maps can be applied to a different iterable, for example to a text string:

19.3 Filters

The built-in function filter(fun, iterable) is somewhat similar to map(fun, iterable). The difference is that the function fun should return True or False. It is applied to all items of the iterable, and all items for which it does not return True are left out. Analogously to map, the function filter returns a Filter Object which needs to be cast to an iterable. Here is an example where the filter picks from a list of text strings all items which begin with a capital letter:

Again, the same can be done using list comprehension, which moreover is a bit simpler because it does not involve the cast to list:

19.4 Function reduce()

Function reduce(fun, iterable) from the functools module is very handy for cumulative operations with lists. By a cumulative operation we mean an operation which takes all list items, one by one, and does something with them. For example - adding all items, multiplying all items, concatenating all items (if they are text strings), etc. The following example adds all items in a list:

The example is not self-explanatory, so let’s go through it one step at a time:

1. The first x and y to go into the lambda are the first two items in the list: x=1, y=2, and x+y yields 3.
2. The next x and y to go into the lambda are the last result 3 and the next list item (also 3): x=3, y=3, and x+y = 6.
3. The next x and y to go into the lambda are the last result 6 and the next list item 4: x=6, y=4, and x+y = 10.
4. Last step: x=10, y=5, and x+y = 10.

Of course, the same can be done the hard way:

However, using reduce combined with an anonymous function shows that you have a different level of knowledge of Python. Here is one more example:

Example 2: Logical product of a Boolean list

Imagine that we have a list L of Boolean values (True or False). By a logical product of all items we mean L[0] and L[1] and ... and L[n-1]. The result will be True if all values in the list are True, and False otherwise. This can be nicely done with the help of reduce:

And, let’s show one last example:

Example 3: Area below the graph of a function

The area below the graph of a function f(x) can be obtained by integrating f(x) between the points a and b . While integration is an advanced topic in calculus, one does not need any calculus to do this in Python. One just needs to know how to calculate the area of a rectangle. Then one constructs thin columns under the graph of the function, and adds their areas together - that’s it! BTW, the width of the individual columns is h = (b - a)∕n and the array of function values at the grid points is f(X).

Here is the code for a sample function f(x) = sin(x) in the interval (0,π):

The exact value of the area is 2. So, obtaining 1.9835235375094544 with just 10 columns is not bad at all. When one increases the number of columns, the result becomes more accurate: With n = 100 one obtains 1.9998355038874436, and increasing n even further to n = 1000 yields 1.9999983550656637.

19.5 Shallow and deep copying

Before we start talking about shallow and deep copies, recall that copying immutable objects (numerical variables, text strings, tuples, ...) is easy because just assigning them to a new variable creates a new copy automatically. (Mutability was discussed in Subsection 7.33.)

For illustration, in the following example we assign 5 to the variable a, then assign a to b, and change a afterwards. Note that b is still 5:

We will see the same for text strings which are also immutable:

But the situation is completely different when we work with a mutable object (list, dictionary, set, class, ...):

As you can see, changing object a caused the same change to occur in object b. The reason is that objects a and b are at the same place in the memory, so altering one will automatically alter the other (this was explained already in Subsection 7.13).

Shallow copying

The difference between shallow and deep copying is only relevant for compound objects (such as lists that contain other lists, lists which contain instances of classes, etc.). A shallow copy with create a new object, but it will not create recursive copies of the other objects which are embedded in it. In other words, shallow copy is only one level deep. This is best illustrated on an example.

Let us create a list of lists named a, make a shallow copy b, and then alter one of the lists contained in the original list a. As a result, the corresponding list contained in the shallow copy b will change as well:

Copying the list a via

leads to the same result (it also creates a shallow copy). For completeness, a shallow copy of a dictionary a can be obtained via

and a shallow copy of a set a can be obtained by typing:

Deep copying

In contrast to shallow copying, a deep copy constructs a new compound object and then, recursively, inserts into it of the copies of the objects found in the original. The best way to do this is to use the copy module in the Python standard library. Let’s return to the previous example with the list of lists, replacing the shallow copy with a deep one:

As you can see, this time altering the object a did not cause any changes in the object b. Finally, let us remark that the copy module also provides a function copy to create shallow copies of objects.

Table of Contents

• Preface
• 1. Introduction
• 2. Using Python as a Scientific Calculator
• 3. Drawing, Plotting, and Data Visualization with Matplotlib
• 4. Working with Text Strings
• 5. Variables and Types
• 6. Boolean Values, Functions, Expressions, and Variables
• 7. Lists, Tuples, Dictionaries, and Sets
• 8. Functions
• 9. The ’For’ Loop
• 10. Conditions
• 11. The ’While’ Loop
• 12. Exceptions
• 13. File Operations
• 14. Object-Oriented Programming I – Introduction
• 15. Object-Oriented Programming II – Class Inheritance
• 16. Object-Oriented Programming III – Advanced Aspects
• 17. Recursion
• 18. Decorators
• 19. Selected Advanced Topics