7 Lists, Tuples, Dictionaries, and Sets

7.1 Objectives

In this section you will learn:

• How to work with lists, tuples, dictionaries, and sets.
• Important properties of these data structures.
• How to use methods of these classes to operate with them efficiently.
• The difference between mutable and immutable objects in Python.

7.2 Why learn about lists?

Lists are an extremely popular data structure in Python, and the language provides a wide range of functionality to work with them. In Section 5 we said that a variable is like a container which can hold a text string, a numerical value, a Boolean, and other things. Using the same language, a list is like a cargo train where these containers can be loaded one after another. The train can be as long as one needs, and its length can even change at runtime. One can use a list to store a huge amount of data, and process it using many built-in tools that we are going to explore in this section.

7.3 Creating a list

An empty list named, for example, L is created as follows:

A nonempty list is created by listing comma-separated items and enclosing them in square brackets. Let’s create a list L2 of integers:

Here is a list cls with the names of ten students:

Lists can contain virtually anything, including other lists. Here is a list named pts which contains the coordinates of four points in the XY plane:

And last, items in a list can have different types:

As usual, Python gives you a lot of freedom – do not abuse it. Instead of throwing all data into one list, it pays off to have the data better organized.

7.4 Important properties of lists

In contrast to dictionaries and sets, lists can contain repeated items. This is a valid list:

Further, lists are ordered. In other words, two lists containing the same items but ordered differently are not the same:

7.5 Measuring the length of a list

You already know from Subsection 4.5 how to measure the length of text strings using the built-in function len. The same function can be used to measure the length of lists:

Lists have many other similarities to text strings – they can be added together, multiplied with integers, their items can be accessed via indices, they can be sliced, etc. You will see all that later in this section.

7.6 Appending new items to lists

The list method append will append a new item to the end of a list:

The original list can be empty:

As mentioned earlier, a list can contain items of different types, including other lists:

And last, items can be appended to the end of a list without using the method append:

Here, [name] is a one-item list containing the text string name. The line names += [name] adds the list [name] to the list names. We are going to talk more about adding lists in the next subsection.

7.7 Adding lists

You already know from Subsection 4.20 that text strings can be added using the operator + just like numbers. Lists can be added in the same way:

A list can be extended by adding another list to it using the operator +=:

7.8 Adding is not appending

Novice programmers sometimes by mistake append a list Y to an existing list X instead of adding Y to X. But this leads to a completely wrong result:

The correct way to add the list Y to the list X is:

7.9 Multiplying lists with integers

Do you still remember from Subsection 4.21 that a text string can be made N times longer by multiplying it with a positive integer N? Multiplication of lists with positive integers works analogously:

The operator *= can be used to extend the list in place:

7.10 Parsing lists with the for loop

We met the for loop for the first time in Subsection 4.22 where it was used to parse text strings one character at a time. Analogously, it can be used to parse lists one item at a time:

The for loop will be discussed in more detail in Section 9.

7.11 Accessing list items via their indices

In Subsection 4.24 you saw how individual characters in a text string can be accessed using their indices. Individual items in a list are accessed exactly in the same way:

7.12 Slicing lists

We are not going to discuss list slicing in detail because lists are sliced in exactly the same way as text strings. This was explained in Subsection 4.29. But showing one small example cannot hurt:

7.13 Creating a copy of a list – the wrong way and the right way

To create a new copy Lnew of a list L, it is not enough to just assign it to a new variable

the way it works for text strings. Namely, this does not copy the list - it only makes Lnew point to the same place in memory as L. We will talk about this in more detail when discussing mutability of lists in Subsection 7.34. For now let’s show what surprise one may get when "copying" a list in this way.

The code below "creates a copy" Lnew of a list L by typing Lnew = L. Then it appends a new item to the end of Lnew. But as you will see, the new item also appears in the original list L!

The correct way to create a copy Lnew of a list L is by slicing:

Let us make this tiny change in the code above, and you will see that the undesired item 4 will not appear in the original list L anymore:

7.14 Popping list items by index

Class list has a method pop which removes the last item from the list and returns it for further use:

When called with an index, the method will pop the corresponding item instead of the last one. Since indices start from 0, using the index 0 will pop the first item:

For illustration, let’s also pop the second item using index 1:

7.15 Deleting list items by index

Typing del L[i] will delete item with index i from list L and destroy it. For illustration, let’s delete the first item from the list team:

7.16 Checking if an item is in a list

In Subsection 4.35 you learned that typing substr in txt will return True if the substring substr is present in the text string txt, and False otherwise. The keyword in can be used in the same way to check whether an item is present in a list:

Now let’s check for ’Xander’ and store the result in a Boolean variable found:

7.17 Locating an item in a list

Now, let’s say that we need to delete Brian from the team. You can see the list of names in the code below, but in reality you would not see it – you would not know where the name ’Brian’ is located. This means that we first need to locate the name ’Brian’ using the method index, and then delete the name using the keyword del:

If the list item is not found, the method index will throw an error:

7.18 Finding and deleting an item from a list

A safe way to delete an item from a list is to first check whether the item is there:

7.19 Finding and deleting all occurrences of an item

Only a minor change to the previous program is needed to find and delete all occurrences of a given item from a list:

7.20 Counting occurrences of items

Class list has a method count to count the occurrences of an item in the list:

7.21 Inserting list items at arbitrary positions

New item x can be inserted to a list L at position n by typing L.insert(n, x). In other words, after the operation, item x will have index n in the list L. For illustration, let us insert the name ’Daniel’ at the fourth position (index 3) to a list of names team:

7.22 Sorting a list in place

Class list provides a method sort to sort the list in place. By in place we mean that the original list is changed. If you need to create a sorted copy of a list, see Subsection 7.23.

In case you are aware of the existence of various sorting algorithms (BubbleSort, QuickSort, MergeSort, InsertionSort, ...), Python uses an algorithm called TimSort. This is a hybrid sorting algorithm derived from MergeSort and InsertionSort, designed to perform well on many kinds of real-world data. It was invented by Tim Peters in 2002 for use in the Python programming language. Here is an example:

7.23 Creating sorted copy of a list

The obvious solution to this task is to create a copy of the list and then sort it using the sort method:

However, Python makes this easier by providing a built-in function sorted. This function creates a copy of the original list, sorts it, and returns it, leaving the original list unchanged:

7.24 Using key functions in sorting

Both the method sort and the function sorted accept an optional argument key. This is a function to be applied to all list items prior to sorting. For example, if one wants to disregard the case of characters while sorting a list of text strings, one can define key=str.lower. To illustrate this, first let’s sort a sample list without the key:

And now let’s use key=str.lower:

Alternatively to key=str.lower, one could use any other string method (many were discussed in Section 4).

7.25 Using lambda functions in sorting

The so-called lambda functions are a quick and elegant way to define one’s own sorting keys. Lambda functions will be discussed in detail in Subsection 8.18 but let’s show a simple example now. Let’s say that we want for some reason to sort a list of words according to the last character in each word. The corresponding lambda function which takes a text string and returns its last character has the form

And now the code:

7.26 Reversing a list in place

Class list provides a method reverse to reverse the list in place:

7.27 Creating reversed copy of a list

Analogously to creating a sorted copy of a list (Subsection 7.23), there are two ways to do this. First, one can just create a copy of the list and then reverse it using the reverse method:

Or, one can call the built-in function reversed:

Note: Since this function is designed for more general applications, it is necessary to cast the result to a list.

7.28 Zipping lists

Another useful operation with lists is their zipping. By this we mean taking two lists of the same length and creating a new list of pairs. Python does this using a built-in function zip. Example:

If the lists are not equally long, the items at the end of the longer list are skipped:

7.29 List comprehension

List comprehension is a quick and elegant way to perform some operation with all items in an iterable (text string, list, tuple, dictionary, ...), creating a new list in the process. This technique is genuinely Pythonic and extremely popular in the Python programming community. Let’s begin with a simple example where we want to square all numbers in a list:

The next example converts a list of characters to a list of the corresponding ASCII codes:

Oh, wait – a text string is iterable, so one can use it in the list comprehension directly!

7.30 List comprehension with if statement

It is possible make an if statement part of the list comprehension. Then the operation is only performed on some items in the list while other items are ignored (and left out).

A common application of this type of list comprehension is filtering – this means that one picks selected items from the list without changing them (although changing them is possible as well). As an example, let’s select all even numbers from a list of integers L:

One can use list comprehension to extract from a list of payments all payments which were made in EUR:

Or, one can use list comprehension to extract all integers from a text string:

And as a last example, let’s write a program which will select from list A all items which are not in list B:

7.31 List comprehension with if and else

In its fullest form, list comprehension makes it possible to incorporate conditional (ternary) expressions of the form

Importantly, a conditional expression is not an if-else statement. We will talk about this in more detail in Subsection 10.10. In list comprehension, they can be used as follows:
Let’s demonstrate this on an example where all characters in a list A which are not vowels will be replaced with ’-’:

For better readability, it is possible to enclose the conditional expression in parentheses. This is shown in the next example where vowels are replaced with 1s and all other characters with 0s:

7.32 List comprehension with nested loops

List comprehension can use multiple for statements. Imagine that you have a list of numbers N, list of characters C, and you want to create a list of labels L by combining the numbers and characters together:

And if you also want to add some symbols, such as + and -, you can use a triple-nested loop:

The result of a comprehension does not have to be a list. It can be another iterable, notably a tuple or a dictionary. We will talk about this in Subsections 7.45, 7.46 and 7.53.

7.33 Mutable and immutable objects

As you already know, almost everything in Python is an object. There are two types of objects – mutable and immutable. Immutable objects cannot be changed by functions. In other words, one can pass an immutable object into a function, the function can attempt to alter it, but after the function finishes, the object will remain unchanged.

Text strings (str) are immutable objects. To illustrate what we said above, we will pass a sample text string txt to a function redef which will try to change it:

Clearly the text string txt was not changed. To understand what is going on, let’s print the memory locations of the variables txt and x prior and after the changes. You will see that when txt is passed to the function, the variable x points to the same memory address as txt. But as soon as x is overwritten, its location in the memory changes. Hence, the memory segment where txt is stored remains unaltered – meaning that txt does not change:

Sometimes, novice programmers incorrectly interpret immutability as "the object cannot be changed". But that’s not correct. Sure enough a text string can be changed:

Let’s display the memory location of the variable txt before and after. You will see again that its memory location changes:

This is the true meaning of immutability. Soon you will see that mutable objects can be altered while remaining at the same memory location. But in the meantime, let’s explore additional immutable objects:

Integers (int), real numbers (float), and complex numbers (complex) are immutable. Let’s just show this using a real number. We will again pass it into a function which will try to change it:

Again, let’s print the memory location of the numerical variable prior and after the change, and you will see that it will be different:

At this point you understand immutability. For completeness let’s remark that Booleans (bool) and tuples (tuple) are immutable types as well. Mutable types include lists (list), dictionaries (dict), sets (set) and most custom classes.

In the next subsection we will look at mutability of lists in more detail.

7.34 Mutability of lists

With the understanding of immutability that you have from the previous subsection we can now offer two equivalent definitions of mutability:

1. An object is mutable if it can be changed inside functions.
2. An object is mutable if it can be altered while remaining at the same memory location.

Let us illustrate the first point by passing a list into a function redef which will append a new item to it.

To illustrate the second point, let’s show that appending a new item to a list will not change the list’s location in the memory:

Being able to check where objects are stored in computer memory can help you solve all sorts of mysteries. For example, the following experiment clearly reveals the source of the problem that was described in Subsection 7.13 (creating a copy of a list incorrectly):

As you can see, L and K really are just two names for the same list which is located at the same memory position 0x27c4b90.

7.35 Tuples

Tuples are very similar to lists, with the following differences:

1. Tuples use parentheses (...) where lists use square brackets [...].
2. Tuples cannot be changed in place – they can only be overwritten like text strings.
3. Tuples are immutable – they cannot be changed by functions.

These propertes make tuples perfect to represent sequences of data that does not change – such as the names of weekdays, or names of months:

All items in this tuple are text strings, but a tuple can be heterogeneous – it can contain any combination of text strings, integers, real numbers, other tuples, functions, instances of classes, etc.

Working with tuples is similar to working with lists. Items in a tuple can be accessed via their indices:

One can use negative indices:

Tuples can be sliced like lists:

The length of a tuple is obtained using the function len():

Let’s now discuss the aspects of tuples which are different from lists.

7.36 Read-only property of tuples

If you search the web, you will find numerous discussions about when one should use tuples and when one should use lists.

As a matter of fact, the speed is not a significant factor – in other words, one cannot say that using tuples instead of lists wherever possible will speed up your code.

But one aspect of tuples is beneficial for sure – their read-only property. One cannot append, insert, pop or delete items from tuples:

So, if you have a sequence of data which should not be altered by anyone in the team, use a tuple.

7.37 Immutability of tuples

Tuples are immutable. In this, they are similar to text strings. A tuple cannot be changed by a function. It only can be overwritten, in which case it changes the location in the memory:

For a more detailed discussion of mutable and immutable objects see Subsection 7.33.

7.38 Dictionaries

Sometimes one needs to store objects along with some related information. For example, these "objects" may be people, and we may need to store their phone number, age, address, family status, etc. Or, we may need to store a list of countries along with their GDP. Or a list of used cars with their price.

Well, you know that lists can contain other lists, so you could handle this:

But, this is a poor man’s solution. Searching for items and other operations would be cumbersome. Here is what it would take to find the value (second item) for a given key (first item):

Therefore, Python provides a dedicated data type called dictionary. Under the hood a dictionary is similar to a list of lists, but it has a lot of useful functionality which we will explore in the following subsections. To begin with, one does not need a function findvalue. Instead, one just types

7.39 Creating an empty and nonempty dictionary

An empty dictionary D is defined as follows:

As you already saw, the dictionary for the above example is

Note that dictionaries use curly braces {...} where lists use square brackets [...] and tuples parentheses (...).

The length of the dictionary is the number of key:value pairs. It can be measured using the function len:

7.40 Keys, values, and items

In the last example, the cars were the keys and their prices were the corresponding values. The list of all keys can be obtaining by typing

Similarly, one can also extract the list of all values:

The key:value pairs are called items. Each pair is a two-item tuple. A list of these tuples can be obtained by typing:

7.41 Accessing values using keys

In lists and tuples it is clear which item is first, which one is second, etc. That’s because they are ordered. Then one can access their items using indices, slice them, reverse them, etc.

None of this is possible with dictionaries.

Dictionaries are not ordered, so there is no first item and there is no last item. They do not have indices and they cannot be sliced. Instead, values are accessed using the keys directly. As you already saw before:

In other words, the keys serve as indices.

7.42 Adding, updating, and deleting items

Here is a small phone book:

A new_key:new_value pair can be added to an existing dictionary dict_name by typing dict_name[new_key] = new_value. So, Silly Sam with the phone number 1234567 would be added as follows:

Dictionaries cannot contain repeated keys. Therefore, when adding a key:value pair whose key already exists in the dictionary, no new item will be created. Instead, only the value in the existing key:value pair will be updated:

And last, items can be deleted using the keyword del:

7.43 Checking for keys, values, and items

To check for a given key, one can type key in dict_name:

It is also possible to use the slightly longer version key in dict_name.keys(). To check for a given value, one can type value in dict_name.values():

And finally, one can check for the presence of a given key:value pair by typing (key, value) in dict_name.items():

7.44 Finding all keys for a given value

Note that while there is only one value for every key, multiple different keys may have the same value. Such as in this dictionary where the Toyota and the Subaru have the same price:

Hence one has to parse all items in the dictionary to find all the keys:

But this task has a way more elegant solution which we can show you because you already know list comprehension. Recall that comprehension was discussed in Subsections 7.29 – 7.32.

7.45 Finding all keys for a given value using comprehension

Comprehension for a dictionary D works analogously to the comprehension for a list L, except that instead of for x in L one types for k, v in D.items(). A shorter solution to the task from the previous subsection is:

7.46 Reversing a dictionary using comprehension

Comprehension is a really powerful ally of the Python programmer. Reversing a dictionary is a one-liner:

7.47 Beware of repeated values

If the dictionary contains repeated values, reversing it becomes an ill-defined task with unpredictable results. For illustration, here is the code from the previous subsection again, except we added one more car whose price is 15000:

But wait - where is the Camry?

The answer is that the pair 15000:’Toyota Camry’ was overwritten with the pair 15000:’Subaru Forester’ (you know from Subsection 7.42 that a dictionary cannot contain repeated keys). However, since the items in a dictionary are not ordered, the pair 15000:’Subaru Forester’ could very easily have been overwritten with the pair 15000:’Toyota Camry’. Then not the Camry, but the Forester would be missing. Hence the outcome of the program depends on a concrete implementation of Python, and/or on the order we insert items in our dictionary, neither of which is a good thing.

7.48 Creating a dictionary from two lists

You already know how to decompose a dictionary D into a pair of lists D.keys() and D.values(). This is the reverse task. But it can be solved easily using the zip method you know from Subsection 7.28:

7.49 Reversing a dictionary using zip

Here is an alternative solution to the task from Subsection 7.46 based on zipping:

As you can see, the Camry disappeared again. This is not a fault of the method – the problem is that the dictionary contains repeated values, as we discussed in Subsection 7.47.

7.50 Creating a copy of a dictionary

Dictionaries are mutable objects (see Subsection 7.33) and therefore one could make a mistake trying to create a copy cars2 of cars by typing cars2 = cars. The code below does that:

You can see that when the "copy" cars2 was altered, so was the original dictionary. The reason is that both cars2 and cars point to the same location in the memory.

The correct way to create a copy is to type either cars2 = dict(cars) or cars2 = cars.copy(). Let’s use the former approach in the following example. The example also shows that when cars2 is altered, cars remains unchanged:

7.51 Merging dictionaries

Let’s say that we have two dictionaries such as

and

and we need to merge cars2 into cars. The dict class has a method update for this:

Notice that the price of the Audi is 30000 now – the reason is that the original value in cars was overwritten with the new value from cars2 for the same key.

7.52 Adding dictionaries

Remember how text strings and lists can be added using the + symbol? It would be nice if this worked for dictionaries too, but unfortunately it does not (maybe it will, in a future version of Python). The classical way to add two dictionaries includes two steps:

1. Create a copy of the first dictionary.
2. Merge the other dictionary into it.

Here is an example:

Starting with Python version 3.5, it is possible to use a compact notation {**X, **Y} to add dictionaries X and Y. If there are shared keys, then values from Y will overwrite the corresponding values in X. Let’s illustrate this on our cars:

7.53 Composing dictionaries

This is another nice exercise for dictionary comprehension. Let’s define two sample dictionaries – one English-Spanish,

and one English-French:

Assuming that all keys which are present in en_es are also present in en_fr, one can create the corresponding Spanish-French dictionary as follows:

However, this simple approach will fail with a KeyError if the sets of keys in en_es and en_fr differ. Since the comprehension goes over all keys in en_es, the remedy is to include an additional condition checking that the key also is in en_fr:

7.54 Sets

Do you still remember your Venn diagrams, and set operations such as set union and set intersection? That’s exactly what sets in Python are foor. The implementation of the set data type in Python follows closely their math definition. One needs to remember that:

• They are not ordered.
• They cannot contain repeated items.
• They are mutable.

In the rest of this section we will show you their most important properties, and some interesting things you can do with sets.

7.55 Creating sets

An empty set S can be created by typing

The easiest way to create a nonempty set is to type:

Notice that sets use curly braces, same as dictionaries. You can also pass a list into their constructor:

As you can see, the repeated items were removed. And as a last example, one can initialize a set with a text string, in which case the set will contain all invididual characters without repetition:

As you can see, function len can be used to obtain the number of items in a set.

7.56 Adding and removing items

New element x can be added to a set S by typing S.add(x):

An entire new set N can be added to an existing set S by typing S.update(N):

As you can see, the order of elements in a set can change arbitrarily while performing set operations.

There are two basic ways to remove an element x from a set S. First, typing

However, this will raise KeyError if the element is not in the set. Alternatively, one can also use

which will only remove the element if it’s present. And last, typing

will remove all elements from the set S.

7.57 Popping random elements

Since sets are not ordered like lists, it is not possible to pop the first element, the last element, or an element with a given index. Instead, calling S.pop() will remove and return a random element. The function will throw a KeyError if the set is empty. Here is an example:

To avoid the KeyError, you can always check whether a set is empty before popping an element:

7.58 Checking if an element is in a set

As you would expect, this is done using the keyword in as with lists, tuples, and dictionaries:

7.59 Checking for subsets and supersets

Let’s have three sets

One can check if C is a subset of A (meaning that every element of C is also present in A) by typing:

The same can be done using the symbol <=:

Next, one can check if A is a superset of B (meaning that every element of B is also present in A) by typing:

Equivalently, one can use the symbol >=:

7.60 Intersection and union

Let’s have the same three sets as in the previous subsection:

The intersection of A and B is the set of all elements which are both in A and in B:

Equivalently, one can use the symbol &:

The union of A and B is the set of all elements which are in A or in B (or in both at the same time):

Equivalently, one can use the symbol |:

7.61 Difference and symmetric difference

Here are the three sets from the previous subsection again:

The set difference of A and B is the set of all elements of A which are not in B:

Equivalently, one can use the symbol -:

The symmetric difference of A and B is the set of all elements of A which are not in B, and of all elements of B which are not in A:

Equivalently, one can use the symbol ^:

We will stop here but there is additional set functionality that you can find in the official Python documentation at https://docs.python.org.

7.62 Using a set to remove duplicated items from a list

Let’s close this section with a neat application of sets – converting a list to a set a set can be used to quickly and elegantly remove duplicated items from a list:

One has to keep in mind though that the conversion to a set and back to a list can change the order of items.

---

Table of Contents

• Preface
• 1. Introduction
• 2. Using Python as a Scientific Calculator
• 3. Drawing, Plotting, and Data Visualization with Matplotlib
• 4. Working with Text Strings
• 5. Variables and Types
• 6. Boolean Values, Functions, Expressions, and Variables
• 7. Lists, Tuples, Dictionaries, and Sets
• 8. Functions
• 9. The ’For’ Loop
• 10. Conditions
• 11. The ’While’ Loop
• 12. Exceptions
• 13. File Operations
• 14. Object-Oriented Programming I – Introduction
• 15. Object-Oriented Programming II – Class Inheritance
• 16. Object-Oriented Programming III – Advanced Aspects
• 17. Recursion
• 18. Decorators
• 19. Selected Advanced Topics