Introduction to Python Programming. Section 7. Lists, Tuples, Dictionaries, and Sets
7 Lists, Tuples, Dictionaries, and Sets
7.1 Objectives
In this section you will learn:
- How to work with lists, tuples, dictionaries, and sets.
- Important properties of these data structures.
- How to use methods of these classes to operate with them efficiently.
- The difference between mutable and immutable objects in Python.
7.2 Why learn about lists?
Lists are an extremely popular data structure in Python, and the language provides a wide range of functionality to work with them. In Section 5 we said that a variable is like a container which can hold a text string, a numerical value, a Boolean, and other things. Using the same language, a list is like a cargo train where these containers can be loaded one after another. The train can be as long as one needs, and its length can even change at runtime. One can use a list to store a huge amount of data, and process it using many built-in tools that we are going to explore in this section.
7.3 Creating a list
An empty list named, for example, L is created as follows:
A nonempty list is created by listing comma-separated items and enclosing them in square
brackets. Let’s create a list L2 of integers:
Here is a list cls with the names of ten students:
Lists can contain virtually anything, including other lists. Here is a list named pts which
contains the coordinates of four points in the XY plane:
And last, items in a list can have different types:
As usual, Python gives you a lot of freedom – do not abuse it. Instead of throwing all data into one list, it pays off to have the data better organized.
7.4 Important properties of lists
In contrast to dictionaries and sets, lists can contain repeated items. This is a valid
list:
Further, lists are ordered. In other words, two lists containing the same items but ordered
differently are not the same:
7.5 Measuring the length of a list
You already know from Subsection 4.5 how to measure the length of text strings using
the built-in function len. The same function can be used to measure the length of
lists:
Lists have many other similarities to text strings – they can be added together, multiplied with integers, their items can be accessed via indices, they can be sliced, etc. You will see all that later in this section.
7.6 Appending new items to lists
The list method append will append a new item to the end of a list:
The original list can be empty:
As mentioned earlier, a list can contain items of different types, including other
lists:
And last, items can be appended to the end of a list without using the method
append:
Here, [name] is a one-item list containing the text string name. The line names += [name] adds the list [name] to the list names. We are going to talk more about adding lists in the next subsection.
7.7 Adding lists
You already know from Subsection 4.20 that text strings can be added using the operator +
just like numbers. Lists can be added in the same way:
A list can be extended by adding another list to it using the operator +=:
7.8 Adding is not appending
Novice programmers sometimes by mistake append a list Y to an existing list X instead of
adding Y to X. But this leads to a completely wrong result:
The correct way to add the list Y to the list X is:
7.9 Multiplying lists with integers
Do you still remember from Subsection 4.21 that a text string can be made N times longer by
multiplying it with a positive integer N? Multiplication of lists with positive integers works
analogously:
The operator *= can be used to extend the list in place:
7.10 Parsing lists with the for loop
We met the for loop for the first time in Subsection 4.22 where it was used to parse text
strings one character at a time. Analogously, it can be used to parse lists one item at a
time:
The for loop will be discussed in more detail in Section 9.
7.11 Accessing list items via their indices
In Subsection 4.24 you saw how individual characters in a text string can be accessed using
their indices. Individual items in a list are accessed exactly in the same way:
Second item: a
Third item: s
Fourth item: t
7.12 Slicing lists
We are not going to discuss list slicing in detail because lists are sliced in exactly the same
way as text strings. This was explained in Subsection 4.29. But showing one small example
cannot hurt:
L[:6] = [’c’, ’o’, ’f’, ’f’, ’e’, ’e’]
L[6:] = [’m’, ’a’, ’k’, ’e’, ’r’]
L[:] = [’c’, ’o’, ’f’, ’f’, ’e’, ’e’, ’m’, ’a’, ’k’, ’e’, ’r’]
L[::2] = [’c’, ’f’, ’e’, ’m’, ’k’, ’r’]
L[5::-1] = [’e’, ’e’, ’f’, ’f’, ’o’, ’c’]
7.13 Creating a copy of a list – the wrong way and the right way
To create a new copy Lnew of a list L, it is not enough to just assign it to a new
variable
the way it works for text strings. Namely, this does not copy the list - it only makes Lnew point to the same place in memory as L. We will talk about this in more detail when discussing mutability of lists in Subsection 7.34. For now let’s show what surprise one may get when "copying" a list in this way.
The code below "creates a copy" Lnew of a list L by typing Lnew = L. Then it appends a
new item to the end of Lnew. But as you will see, the new item also appears in the original
list L!
L = [1, 2, 3, 4]
The correct way to create a copy Lnew of a list L is by slicing:
Let us make this tiny change in the code above, and you will see that the undesired item 4
will not appear in the original list L anymore:
L = [1, 2, 3]
7.14 Popping list items by index
Class list has a method pop which removes the last item from the list and returns it for
further use:
When called with an index, the method will pop the corresponding item instead of the last
one. Since indices start from 0, using the index 0 will pop the first item:
For illustration, let’s also pop the second item using index 1:
7.15 Deleting list items by index
Typing del L[i] will delete item with index i from list L and destroy it. For illustration,
let’s delete the first item from the list team:
7.16 Checking if an item is in a list
In Subsection 4.35 you learned that typing substr in txt will return True if
the substring substr is present in the text string txt, and False otherwise. The
keyword in can be used in the same way to check whether an item is present in a
list:
Now let’s check for ’Xander’ and store the result in a Boolean variable found:
7.17 Locating an item in a list
Now, let’s say that we need to delete Brian from the team. You can see the list of names in
the code below, but in reality you would not see it – you would not know where
the name ’Brian’ is located. This means that we first need to locate the name
’Brian’ using the method index, and then delete the name using the keyword
del:
If the list item is not found, the method index will throw an error:
7.18 Finding and deleting an item from a list
A safe way to delete an item from a list is to first check whether the item is there:
7.19 Finding and deleting all occurrences of an item
Only a minor change to the previous program is needed to find and delete all occurrences of
a given item from a list:
7.20 Counting occurrences of items
Class list has a method count to count the occurrences of an item in the list:
7.21 Inserting list items at arbitrary positions
New item x can be inserted to a list L at position n by typing L.insert(n, x). In other
words, after the operation, item x will have index n in the list L. For illustration, let
us insert the name ’Daniel’ at the fourth position (index 3) to a list of names
team:
7.22 Sorting a list in place
Class list provides a method sort to sort the list in place. By in place we mean that the original list is changed. If you need to create a sorted copy of a list, see Subsection 7.23.
In case you are aware of the existence of various sorting algorithms (BubbleSort,
QuickSort, MergeSort, InsertionSort, ...), Python uses an algorithm called TimSort. This is a
hybrid sorting algorithm derived from MergeSort and InsertionSort, designed to perform
well on many kinds of real-world data. It was invented by Tim Peters in 2002 for use in the
Python programming language. Here is an example:
7.23 Creating sorted copy of a list
The obvious solution to this task is to create a copy of the list and then sort it using the sort
method:
However, Python makes this easier by providing a built-in function sorted. This function
creates a copy of the original list, sorts it, and returns it, leaving the original list
unchanged:
7.24 Using key functions in sorting
Both the method sort and the function sorted accept an optional argument key.
This is a function to be applied to all list items prior to sorting. For example, if one
wants to disregard the case of characters while sorting a list of text strings, one can
define key=str.lower. To illustrate this, first let’s sort a sample list without the
key:
And now let’s use key=str.lower:
Alternatively to key=str.lower, one could use any other string method (many were discussed in Section 4).
7.25 Using lambda functions in sorting
The so-called lambda functions are a quick and elegant way to define one’s own
sorting keys. Lambda functions will be discussed in detail in Subsection 8.18 but
let’s show a simple example now. Let’s say that we want for some reason to sort
a list of words according to the last character in each word. The corresponding
lambda function which takes a text string and returns its last character has the form
7.26 Reversing a list in place
Class list provides a method reverse to reverse the list in place:
7.27 Creating reversed copy of a list
Analogously to creating a sorted copy of a list (Subsection 7.23), there are two ways to do
this. First, one can just create a copy of the list and then reverse it using the reverse
method:
Or, one can call the built-in function reversed:
Note: Since this function is designed for more general applications, it is necessary to cast the result to a list.
7.28 Zipping lists
Another useful operation with lists is their zipping. By this we mean taking two lists of the
same length and creating a new list of pairs. Python does this using a built-in function zip.
Example:
If the lists are not equally long, the items at the end of the longer list are skipped:
7.29 List comprehension
List comprehension is a quick and elegant way to perform some operation with all items in
an iterable (text string, list, tuple, dictionary, ...), creating a new list in the process. This
technique is genuinely Pythonic and extremely popular in the Python programming
community. Let’s begin with a simple example where we want to square all numbers in a
list:
The next example converts a list of characters to a list of the corresponding ASCII
codes:
Oh, wait – a text string is iterable, so one can use it in the list comprehension directly!
7.30 List comprehension with if statement
It is possible make an if statement part of the list comprehension. Then the operation is only
performed on some items in the list while other items are ignored (and left out).
A common application of this type of list comprehension is filtering – this means that one
picks selected items from the list without changing them (although changing them is
possible as well). As an example, let’s select all even numbers from a list of integers
L:
One can use list comprehension to extract from a list of payments all payments which were
made in EUR:
Or, one can use list comprehension to extract all integers from a text string:
And as a last example, let’s write a program which will select from list A all items which are
not in list B:
7.31 List comprehension with if and else
In its fullest form, list comprehension makes it possible to incorporate conditional (ternary)
expressions of the form
For better readability, it is possible to enclose the conditional expression in parentheses. This
is shown in the next example where vowels are replaced with 1s and all other characters with
0s:
7.32 List comprehension with nested loops
List comprehension can use multiple for statements. Imagine that you have a list of
numbers N, list of characters C, and you want to create a list of labels L by combining the
numbers and characters together:
And if you also want to add some symbols, such as + and -, you can use a triple-nested
loop:
The result of a comprehension does not have to be a list. It can be another iterable, notably a tuple or a dictionary. We will talk about this in Subsections 7.45, 7.46 and 7.53.
7.33 Mutable and immutable objects
As you already know, almost everything in Python is an object. There are two types of objects
– mutable and immutable. Immutable objects cannot be changed by functions. In other words,
one can pass an immutable object into a function, the function can attempt to alter it, but after
the function finishes, the object will remain unchanged.
Text strings (str) are immutable objects. To illustrate what we said above, we will pass a
sample text string txt to a function redef which will try to change it:
Clearly the text string txt was not changed. To understand what is going on, let’s print the
memory locations of the variables txt and x prior and after the changes. You will see that
when txt is passed to the function, the variable x points to the same memory address as
txt. But as soon as x is overwritten, its location in the memory changes. Hence, the
memory segment where txt is stored remains unaltered – meaning that txt does not
change:
Sometimes, novice programmers incorrectly interpret immutability as "the object cannot be
changed". But that’s not correct. Sure enough a text string can be changed:
Let’s display the memory location of the variable txt before and after. You will see again that
its memory location changes:
This is the true meaning of immutability. Soon you will see that mutable objects can be
altered while remaining at the same memory location. But in the meantime, let’s explore
additional immutable objects:
Integers (int), real numbers (float), and complex numbers (complex) are immutable. Let’s
just show this using a real number. We will again pass it into a function which will try to
change it:
Again, let’s print the memory location of the numerical variable prior and after the change,
and you will see that it will be different:
At this point you understand immutability. For completeness let’s remark that Booleans
(bool) and tuples (tuple) are immutable types as well. Mutable types include lists (list),
dictionaries (dict), sets (set) and most custom classes.
In the next subsection we will look at mutability of lists in more detail.
7.34 Mutability of lists
With the understanding of immutability that you have from the previous subsection we can now offer two equivalent definitions of mutability:
- An object is mutable if it can be changed inside functions.
- An object is mutable if it can be altered while remaining at the same memory location.
Let us illustrate the first point by passing a list into a function redef which will append a new
item to it.
To illustrate the second point, let’s show that appending a new item to a list will not change
the list’s location in the memory:
Being able to check where objects are stored in computer memory can help you solve
all sorts of mysteries. For example, the following experiment clearly reveals the
source of the problem that was described in Subsection 7.13 (creating a copy of a list
incorrectly):
As you can see, L and K really are just two names for the same list which is located at the same memory position 0x27c4b90.
7.35 Tuples
Tuples are very similar to lists, with the following differences:
- Tuples use parentheses (...) where lists use square brackets [...].
- Tuples cannot be changed in place – they can only be overwritten like text strings.
- Tuples are immutable – they cannot be changed by functions.
These propertes make tuples perfect to represent sequences of data that does not change –
such as the names of weekdays, or names of months:
All items in this tuple are text strings, but a tuple can be heterogeneous – it can contain any combination of text strings, integers, real numbers, other tuples, functions, instances of classes, etc.
Working with tuples is similar to working with lists. Items in a tuple can be accessed via
their indices:
Second month: February
Third month: March
One can use negative indices:
Last month: December
Tuples can be sliced like lists:
The length of a tuple is obtained using the function len():
Let’s now discuss the aspects of tuples which are different from lists.
7.36 Read-only property of tuples
If you search the web, you will find numerous discussions about when one should use tuples and when one should use lists.
As a matter of fact, the speed is not a significant factor – in other words, one cannot say that using tuples instead of lists wherever possible will speed up your code.
But one aspect of tuples is beneficial for sure – their read-only property. One cannot
append, insert, pop or delete items from tuples:
AttributeError: ’tuple’ object has no attribute ’append’
So, if you have a sequence of data which should not be altered by anyone in the team, use a tuple.
7.37 Immutability of tuples
Tuples are immutable. In this, they are similar to text strings. A tuple cannot be changed by a
function. It only can be overwritten, in which case it changes the location in the
memory:
For a more detailed discussion of mutable and immutable objects see Subsection 7.33.
7.38 Dictionaries
Sometimes one needs to store objects along with some related information. For example,
these "objects" may be people, and we may need to store their phone number, age, address,
family status, etc. Or, we may need to store a list of countries along with their GDP. Or a list
of used cars with their price.
Well, you know that lists can contain other lists, so you could handle this:
But, this is a poor man’s solution. Searching for items and other operations would be
cumbersome. Here is what it would take to find the value (second item) for a given key (first
item):
Therefore, Python provides a dedicated data type called dictionary. Under the hood a
dictionary is similar to a list of lists, but it has a lot of useful functionality which we will
explore in the following subsections. To begin with, one does not need a function
findvalue. Instead, one just types
7.39 Creating an empty and nonempty dictionary
An empty dictionary D is defined as follows:
As you already saw, the dictionary for the above example is
Note that dictionaries use curly braces {...} where lists use square brackets [...] and
tuples parentheses (...).
The length of the dictionary is the number of key:value pairs. It can be measured using the
function len:
7.40 Keys, values, and items
In the last example, the cars were the keys and their prices were the corresponding values. The
list of all keys can be obtaining by typing
Similarly, one can also extract the list of all values:
The key:value pairs are called items. Each pair is a two-item tuple. A list of these tuples can
be obtained by typing:
7.41 Accessing values using keys
In lists and tuples it is clear which item is first, which one is second, etc. That’s because they
are ordered. Then one can access their items using indices, slice them, reverse them, etc.
None of this is possible with dictionaries.
Dictionaries are not ordered, so there is no first item and there is no last item. They do not
have indices and they cannot be sliced. Instead, values are accessed using the keys directly.
As you already saw before:
In other words, the keys serve as indices.
7.42 Adding, updating, and deleting items
Here is a small phone book:
A new_key:new_value pair can be added to an existing dictionary dict_name by typing
dict_name[new_key] = new_value. So, Silly Sam with the phone number 1234567
would be added as follows:
Dictionaries cannot contain repeated keys. Therefore, when adding a key:value pair whose
key already exists in the dictionary, no new item will be created. Instead, only the value in
the existing key:value pair will be updated:
And last, items can be deleted using the keyword del:
7.43 Checking for keys, values, and items
To check for a given key, one can type key in dict_name:
It is also possible to use the slightly longer version key in dict_name.keys(). To check
for a given value, one can type value in dict_name.values():
And finally, one can check for the presence of a given key:value pair by typing (key,
value) in dict_name.items():
7.44 Finding all keys for a given value
Note that while there is only one value for every key, multiple different keys may have the
same value. Such as in this dictionary where the Toyota and the Subaru have the same
price:
Hence one has to parse all items in the dictionary to find all the keys:
But this task has a way more elegant solution which we can show you because you already know list comprehension. Recall that comprehension was discussed in Subsections 7.29 – 7.32.
7.45 Finding all keys for a given value using comprehension
Comprehension for a dictionary D works analogously to the comprehension for a list L,
except that instead of for x in L one types for k, v in D.items(). A shorter
solution to the task from the previous subsection is:
7.46 Reversing a dictionary using comprehension
Comprehension is a really powerful ally of the Python programmer. Reversing a dictionary is
a one-liner:
7.47 Beware of repeated values
If the dictionary contains repeated values, reversing it becomes an ill-defined task with
unpredictable results. For illustration, here is the code from the previous subsection again,
except we added one more car whose price is 15000:
But wait - where is the Camry?
The answer is that the pair 15000:’Toyota Camry’ was overwritten with the pair 15000:’Subaru Forester’ (you know from Subsection 7.42 that a dictionary cannot contain repeated keys). However, since the items in a dictionary are not ordered, the pair 15000:’Subaru Forester’ could very easily have been overwritten with the pair 15000:’Toyota Camry’. Then not the Camry, but the Forester would be missing. Hence the outcome of the program depends on a concrete implementation of Python, and/or on the order we insert items in our dictionary, neither of which is a good thing.
7.48 Creating a dictionary from two lists
You already know how to decompose a dictionary D into a pair of lists D.keys() and
D.values(). This is the reverse task. But it can be solved easily using the zip method you
know from Subsection 7.28:
7.49 Reversing a dictionary using zip
Here is an alternative solution to the task from Subsection 7.46 based on zipping:
As you can see, the Camry disappeared again. This is not a fault of the method – the problem is that the dictionary contains repeated values, as we discussed in Subsection 7.47.
7.50 Creating a copy of a dictionary
Dictionaries are mutable objects (see Subsection 7.33) and therefore one could make a mistake
trying to create a copy cars2 of cars by typing cars2 = cars. The code below does
that:
You can see that when the "copy" cars2 was altered, so was the original dictionary. The reason is that both cars2 and cars point to the same location in the memory.
The correct way to create a copy is to type either cars2 = dict(cars) or cars2 =
cars.copy(). Let’s use the former approach in the following example. The example also
shows that when cars2 is altered, cars remains unchanged:
7.51 Merging dictionaries
Let’s say that we have two dictionaries such as
and
and we need to merge cars2 into cars. The dict class has a method update for
this:
Notice that the price of the Audi is 30000 now – the reason is that the original value in cars was overwritten with the new value from cars2 for the same key.
7.52 Adding dictionaries
Remember how text strings and lists can be added using the + symbol? It would be nice if this worked for dictionaries too, but unfortunately it does not (maybe it will, in a future version of Python). The classical way to add two dictionaries includes two steps:
- Create a copy of the first dictionary.
- Merge the other dictionary into it.
Here is an example:
Starting with Python version 3.5, it is possible to use a compact notation {**X, **Y} to add
dictionaries X and Y. If there are shared keys, then values from Y will overwrite the
corresponding values in X. Let’s illustrate this on our cars:
7.53 Composing dictionaries
This is another nice exercise for dictionary comprehension. Let’s define two sample
dictionaries – one English-Spanish,
and one English-French:
Assuming that all keys which are present in en_es are also present in en_fr, one can create
the corresponding Spanish-French dictionary as follows:
However, this simple approach will fail with a KeyError if the sets of keys in
en_es and en_fr differ. Since the comprehension goes over all keys in en_es,
the remedy is to include an additional condition checking that the key also is in
en_fr:
7.54 Sets
Do you still remember your Venn diagrams, and set operations such as set union and set intersection? That’s exactly what sets in Python are foor. The implementation of the set data type in Python follows closely their math definition. One needs to remember that:
- They are not ordered.
- They cannot contain repeated items.
- They are mutable.
In the rest of this section we will show you their most important properties, and some interesting things you can do with sets.
7.55 Creating sets
An empty set S can be created by typing
The easiest way to create a nonempty set is to type:
Notice that sets use curly braces, same as dictionaries. You can also pass a list into their
constructor:
As you can see, the repeated items were removed. And as a last example, one can initialize a
set with a text string, in which case the set will contain all invididual characters without
repetition:
As you can see, function len can be used to obtain the number of items in a set.
7.56 Adding and removing items
New element x can be added to a set S by typing S.add(x):
An entire new set N can be added to an existing set S by typing S.update(N):
As you can see, the order of elements in a set can change arbitrarily while performing set operations.
There are two basic ways to remove an element x from a set S. First, typing
However, this will raise KeyError if the element is not in the set. Alternatively, one can also
use
which will only remove the element if it’s present. And last, typing
will remove all elements from the set S.
7.57 Popping random elements
Since sets are not ordered like lists, it is not possible to pop the first element, the last element,
or an element with a given index. Instead, calling S.pop() will remove and return a
random element. The function will throw a KeyError if the set is empty. Here is an
example:
To avoid the KeyError, you can always check whether a set is empty before popping an
element:
7.58 Checking if an element is in a set
As you would expect, this is done using the keyword in as with lists, tuples, and
dictionaries:
7.59 Checking for subsets and supersets
Let’s have three sets
One can check if C is a subset of A (meaning that every element of C is also present in A) by
typing:
The same can be done using the symbol <=:
Next, one can check if A is a superset of B (meaning that every element of B is also present in
A) by typing:
Equivalently, one can use the symbol >=:
7.60 Intersection and union
Let’s have the same three sets as in the previous subsection:
The intersection of A and B is the set of all elements which are both in A and in
B:
Equivalently, one can use the symbol &:
The union of A and B is the set of all elements which are in A or in B (or in both at the same
time):
Equivalently, one can use the symbol |:
7.61 Difference and symmetric difference
Here are the three sets from the previous subsection again:
The set difference of A and B is the set of all elements of A which are not in B:
Equivalently, one can use the symbol -:
The symmetric difference of A and B is the set of all elements of A which are not in B, and of
all elements of B which are not in A:
Equivalently, one can use the symbol ^:
We will stop here but there is additional set functionality that you can find in the official Python documentation at https://docs.python.org.
7.62 Using a set to remove duplicated items from a list
Let’s close this section with a neat application of sets – converting a list to a set a set can be
used to quickly and elegantly remove duplicated items from a list:
One has to keep in mind though that the conversion to a set and back to a list can change the order of items.
Table of Contents
- Preface
- 1. Introduction
- 2. Using Python as a Scientific Calculator
- 3. Drawing, Plotting, and Data Visualization with Matplotlib
- 4. Working with Text Strings
- 5. Variables and Types
- 6. Boolean Values, Functions, Expressions, and Variables
- 7. Lists, Tuples, Dictionaries, and Sets
- 8. Functions
- 9. The ’For’ Loop
- 10. Conditions
- 11. The ’While’ Loop
- 12. Exceptions
- 13. File Operations
- 14. Object-Oriented Programming I – Introduction
- 15. Object-Oriented Programming II – Class Inheritance
- 16. Object-Oriented Programming III – Advanced Aspects
- 17. Recursion
- 18. Decorators
- 19. Selected Advanced Topics