Introduction to Python Programming. Section 19. Selected Advanced Topics
The goal of this section is to introduce selected advanced programming techniques in Python. Since Python is a modern language which is still evolving, we recommend that after reading about a technique here, you also search it on the web for possible updates.
You will learn about:
- Maps and filters.
- The built-in function reduce.
- Creating shallow and deep copies.
You already know list comprehension well from Subsections 7.29 - 7.32. You also know that comprehension works for any iterable, not only for lists. The difference between a map and a comprehension is rather subtle, and many programmers prefer comprehension for being more Pythonic. But it is a good idea to learn about maps anyway because there are many other people who use them.
The built-in function map(fun, iterable) applies the function fun to each item of the iterable (text string, list, tuple etc.). The function fun can be an anonymous lambda expression or a standard function. For anonymous lambda expressions review Subsections 8.18 - 8.26. The result returned by the map function is a Map Object which needs to be cast back to an iterable.
In the following example, the function fun is given as a lambda expression, and the
iterable is a list of numbers [0, 1, ..., 9] created via range(10):
Let’s take a minute to do the same using list comprehension:
You can see that a cast to list is not needed in this case. As another example, let’s show that
map can use any function, not only lambda expressions:
And last, let’s show that maps can be applied to a different iterable, for example to a text
The built-in function filter(fun, iterable) is somewhat similar to map(fun,
iterable). The difference is that the function fun should return True or False. It is
applied to all items of the iterable, and all items for which it does not return True are left out.
Analogously to map, the function filter returns a Filter Object which needs to be cast to an
iterable. Here is an example where the filter picks from a list of text strings all items which
begin with a capital letter:
Again, the same can be done using list comprehension, which moreover is a bit simpler
because it does not involve the cast to list:
Function reduce(fun, iterable) from the functools module is very handy for
cumulative operations with lists. By a cumulative operation we mean an operation which takes
all list items, one by one, and does something with them. For example - adding all items,
multiplying all items, concatenating all items (if they are text strings), etc. The following
example adds all items in a list:
The example is not self-explanatory, so let’s go through it one step at a time:
- The first x and y to go into the lambda are the first two items in the list: x=1, y=2, and x+y yields 3.
- The next x and y to go into the lambda are the last result 3 and the next list item (also 3): x=3, y=3, and x+y = 6.
- The next x and y to go into the lambda are the last result 6 and the next list item 4: x=6, y=4, and x+y = 10.
- Last step: x=10, y=5, and x+y = 10.
Of course, the same can be done the hard way:
However, using reduce combined with an anonymous function shows that you have a
different level of knowledge of Python. Here is one more example:
Example 2: Logical product of a Boolean list
Imagine that we have a list L of Boolean values (True or False). By a logical product of all
items we mean L and L and ... and L[n-1]. The result will be True if all
values in the list are True, and False otherwise. This can be nicely done with the help of
And, let’s show one last example:
Example 3: Area below the graph of a function
The area below the graph of a function f(x) can be obtained by integrating f(x) between the points a and b . While integration is an advanced topic in calculus, one does not need any calculus to do this in Python. One just needs to know how to calculate the area of a rectangle. Then one constructs thin columns under the graph of the function, and adds their areas together - that’s it! BTW, the width of the individual columns is h = (b - a)∕n and the array of function values at the grid points is f(X).
Here is the code for a sample function f(x) = sin(x) in the interval (0,π):
import numpy as np
# Sample function f(x) = sin(x):
f = np.sin
# End points on the X axis:
a = 0
b = np.pi
# Number of columns:
n = 10
# Equidistant grid between a and b:
X = np.linspace(a, b, n+1)
# Width of the columns:
h = (b - a) / n
# Add the areas of all columns:
import functools as ft
ft.reduce(lambda x, y: x + y*h, f(X))
The exact value of the area is 2. So, obtaining 1.9835235375094544 with just 10 columns is not bad at all. When one increases the number of columns, the result becomes more accurate: With n = 100 one obtains 1.9998355038874436, and increasing n even further to n = 1000 yields 1.9999983550656637.
Before we start talking about shallow and deep copies, recall that copying immutable objects (numerical variables, text strings, tuples, ...) is easy because just assigning them to a new variable creates a new copy automatically. (Mutability was discussed in Subsection 7.33.)
For illustration, in the following example we assign 5 to the variable a, then assign a to b,
and change a afterwards. Note that b is still 5:
We will see the same for text strings which are also immutable:
But the situation is completely different when we work with a mutable object (list, dictionary,
set, class, ...):
As you can see, changing object a caused the same change to occur in object b.
The reason is that objects a and b are at the same place in the memory, so altering
one will automatically alter the other (this was explained already in Subsection
The difference between shallow and deep copying is only relevant for compound objects (such as lists that contain other lists, lists which contain instances of classes, etc.). A shallow copy with create a new object, but it will not create recursive copies of the other objects which are embedded in it. In other words, shallow copy is only one level deep. This is best illustrated on an example.
Let us create a list of lists named a, make a shallow copy b, and then alter one of the lists
contained in the original list a. As a result, the corresponding list contained in the shallow
copy b will change as well:
Copying the list a via
leads to the same result (it also creates a shallow copy). For completeness, a shallow copy of a
dictionary a can be obtained via
and a shallow copy of a set a can be obtained by typing:
In contrast to shallow copying, a deep copy constructs a new compound object and then,
recursively, inserts into it of the copies of the objects found in the original. The best way to do
this is to use the copy module in the Python standard library. Let’s return to the
previous example with the list of lists, replacing the shallow copy with a deep
As you can see, this time altering the object a did not cause any changes in the object b. Finally, let us remark that the copy module also provides a function copy to create shallow copies of objects.
Table of Contents
- 1. Introduction
- 2. Using Python as a Scientific Calculator
- 3. Drawing, Plotting, and Data Visualization with Matplotlib
- 4. Working with Text Strings
- 5. Variables and Types
- 6. Boolean Values, Functions, Expressions, and Variables
- 7. Lists, Tuples, Dictionaries, and Sets
- 8. Functions
- 9. The ’For’ Loop
- 10. Conditions
- 11. The ’While’ Loop
- 12. Exceptions
- 13. File Operations
- 14. Object-Oriented Programming I – Introduction
- 15. Object-Oriented Programming II – Class Inheritance
- 16. Object-Oriented Programming III – Advanced Aspects
- 17. Recursion
- 18. Decorators
- 19. Selected Advanced Topics