Python Collections Module

Lists, tuples, sets and dictionaries handle most jobs, but some jobs come up so often that Python's standard library has ready-made containers for them in the collections module. They make common tasks such as counting, grouping and queuing shorter and clearer.

Counter

A Counter counts how often each value appears. Give it any iterable:

from collections import Counter

words = "the cat and the dog and the bird".split()
counts = Counter(words)
print(counts)
print(counts["the"], counts["fish"])
print(counts.most_common(2))
print(Counter("mississippi").most_common(1))

Counter({'the': 3, 'and': 2, 'cat': 1, 'dog': 1, 'bird': 1})
3 0
[('the', 3), ('and', 2)]
[('i', 4)]

A missing value simply counts as 0 instead of raising an error, and most_common(n) gives the top values in order. Counters can also be added and subtracted.

defaultdict

A defaultdict creates a starting value automatically the first time a missing key is used. It is perfect for grouping:

from collections import defaultdict

students = [("Asha", "Pune"), ("Ravi", "Delhi"), ("Meera", "Pune"), ("John", "Goa")]
by_city = defaultdict(list)
for name, city in students:
    by_city[city].append(name)
print(dict(by_city))
print(by_city["Pune"], by_city["Agra"])

{'Pune': ['Asha', 'Meera'], 'Delhi': ['Ravi'], 'Goa': ['John']}
['Asha', 'Meera'] []

defaultdict(list) starts each new key with an empty list, defaultdict(int) with 0. Without it you would write if city not in groups: groups[city] = [] every time.

namedtuple

A namedtuple is a tuple whose items also have names, which makes code that uses it much easier to read than indexes:

from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p, p.x, p[1])
x, y = p
print(x + y, p._replace(x=10))

Point(x=3, y=4) 3 4
7 Point(x=10, y=4)

deque

A deque (say "deck") is a list that is fast at both ends. It is the right container for queues, and with maxlen it keeps only the most recent items:

from collections import deque

queue = deque(["Asha", "Ravi"])
queue.append("Meera")          # join at the back
first = queue.popleft()        # served from the front
print(first, list(queue))
recent = deque(maxlen=3)
for page in ["home", "news", "sports", "weather", "music"]:
    recent.append(page)
print(list(recent))

Asha ['Ravi', 'Meera']
['sports', 'weather', 'music']

Removing from the front of a normal list with pop(0) gets slow for long lists, because every other item has to move; popleft on a deque stays fast.

Practice exercises

1. Count the letters of a sentence with Counter and print the three most common.
2. Group a list of words by their first letter with defaultdict(list).
3. Make a namedtuple called Book with a title, an author and a year, and print one book's author.
4. Use a deque with maxlen=5 to keep the last five numbers a user types.