The big ideas of Python programming

Everything in Python is an object: numbers, strings, lists, functions, even classes. Objects live in memory. A name like a is just a reference to one of them. The statement b = a doesn't copy anything. It attaches a second label to the same object.

Whether that matters depends on whether the object is mutable. Lists, dicts and sets can change in place, so a change made through b is visible through a. Numbers, strings and tuples are immutable: "changing" y = y + 1 builds a new object and moves the label y to it, leaving every other label alone.

a = [1, 2, 3]
b = a            # same list, two names
b.append(4)
print(a)         # [1, 2, 3, 4]
print(a is b)    # True: one object

Function arguments work the same way. Passing a list to a function hands the function another label for the same list. If the function appends to it, the caller sees the change. If the function rebinds its parameter to a new list, the caller doesn't.

Python's int grows as large as memory allows: 2 ** 100 is exact. A float is different. It's a 64-bit IEEE 754 number: a sign bit, 11 bits of exponent and 52 bits of fraction. That gives about 16 significant decimal digits, which is plenty for most measurements.

The catch is that floats are binary. One tenth in binary is 0.000110011001100…, repeating forever, just as ⅓ repeats in decimal. Python stores the nearest binary fraction, which is slightly more than 0.1. When you add two of these slightly-off numbers, the errors can show:

>>> 0.1 + 0.2
0.30000000000000004
>>> 0.1 + 0.2 == 0.3
False
>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True

This isn't a Python bug. Every language that uses hardware floating point does the same. When you need exact decimal arithmetic, for money for example, use the decimal module or work in integer cents. When you need exact fractions, use fractions.Fraction.

When you write len(x), Python doesn't check whether x is a list. It looks for a method called __len__ on x's type and calls it. x + y calls __add__, x[0] calls __getitem__, and for v in x calls __iter__. These "dunder" (double-underscore) methods are Python's data model, the protocol every object can join.

So your own classes can behave like built-ins. Define __add__ and + works on your objects. Define __iter__ and they work in for loops, list(), sum() and sorted(). Code that only uses those operations works for any type that supports them. This is duck typing, and it is why Python code can be so general.

class Vector:
    def __init__(self, x, y):
        self.x, self.y = x, y
    def __add__(self, other):
        return Vector(self.x + other.x, self.y + other.y)
    def __mul__(self, k):
        return Vector(self.x * k, self.y * k)
    def __iter__(self):
        yield self.x
        yield self.y
    def __eq__(self, other):
        return tuple(self) == tuple(other)
    def __repr__(self):
        return f"Vector({self.x}, {self.y})"

Functions are the main tool of abstraction: give a process a name, decide what goes in and what comes out, and then stop thinking about its insides. A good function can be understood, tested and replaced on its own.

To keep functions independent, each call gets its own local scope. When Python meets a name, it searches four places in order, known as the LEGB rule: the Local scope of the current function, then the scopes of any Enclosing functions, then the module's Global scope, and finally the Built-ins like len and print. The first match wins.

One rule surprises everyone once. If a function assigns to a name anywhere in its body, that name is local throughout the whole function. Python decides this when it compiles the function, before any code runs. The lab's second example shows the resulting UnboundLocalError.

Inner functions remember the enclosing variables they use, even after the outer function has returned. That bundle of function plus remembered variables is a closure, and it's how the third example's counter keeps its count.

Some problems contain smaller copies of themselves. A folder contains folders, an expression contains sub-expressions, and the n-th Fibonacci number is the sum of the two before it. Recursion turns that structure directly into code:

def fib(n):
    if n < 2:          # base case
        return n
    return fib(n - 1) + fib(n - 2)

This is correct but very slow. fib(5) computes fib(3) twice and fib(2) three times, and the waste grows exponentially: fib(30) makes 2.7 million calls. The fix is to remember answers you've already computed. That is memoization, and in Python it's one line:

from functools import cache

@cache
def fib(n):
    ...

Now each fib(k) is computed once, and fib(30) takes 59 calls. Storing subproblem answers is the core of dynamic programming, behind spell checkers, route planners and DNA alignment.

A for loop doesn't need a list. It needs an iterator: an object with a __next__ method that hands over the next item, or raises StopIteration when there are none left. Files, range, dictionaries and database cursors are all iterable.

A generator is the easiest way to write one. Use yield instead of return, or write a generator expression in parentheses. The function pauses at each yield and resumes when the next item is requested. Chain generators together and you get a pipeline that is lazy: nothing is computed until someone asks for it.

from itertools import islice

numbers = range(1, 10**9)
squares = (x * x for x in numbers)
evens   = (s for s in squares if s % 2 == 0)
first5  = list(islice(evens, 5))   # [4, 16, 36, 64, 100]

This finishes instantly, using almost no memory, even though numbers has a billion items. With square brackets (list comprehensions), Python would try to build a billion squares first, which needs tens of gigabytes of memory.

To check whether "kiwi" is in a list of a million fruits, Python compares it with each item in turn, up to a million comparisons. A dict or set does it in about one step, whatever the size.

The trick is a hash function. hash("kiwi") turns the key into a big integer, and the dictionary uses that number to pick a slot in an internal table. To look up a key later, it hashes the key again and goes straight to that slot. Two keys can hash to the same slot, a collision. Then the dictionary probes other slots in a fixed order until it finds the key or an empty slot.

Collisions stay rare because Python keeps the table at most two-thirds full. When it gets fuller, Python allocates a bigger table and reinserts everything. That occasional resize is expensive, but it's rare enough that the average insert stays fast.

stock = {"apple": 3, "kiwi": 12}
stock["fig"] = 7          # insert: hash, find a slot
"kiwi" in stock           # lookup: hash, jump, compare
# True

Searching a sorted list of a million names one by one takes up to a million comparisons. Binary search looks at the middle, throws away the half that can't contain the name, and repeats. It needs at most 20 comparisons, because 2²⁰ is just over a million.

Computer scientists describe this growth with Big-O notation. Linear search is O(n): double the data, double the work. Binary search is O(log n): double the data, one more step. Simple sorting algorithms like insertion sort are O(n²), and the best general sorts are O(n log n). At a million items that's the difference between seconds and days.

import bisect
names.sort()                          # O(n log n), once
i = bisect.bisect_left(names, "Kim")  # O(log n), every lookup

Knowing the cost of built-in operations matters just as much. x in some_list is O(n), while x in some_set is O(1) on average (idea 7). Inserting at the front of a list with insert(0, x) shifts every element. collections.deque does it in constant time.

A test is code that runs your code and checks the answer. With pytest it can be one line: assert is_leap(2024). Tests catch mistakes when you write the code, and keep catching them every time you change it later.

The weakness of example tests is that you only test what you thought of, and the bugs live in what you didn't think of. Property-based testing flips this around. You state a rule that should hold for every input, the library generates hundreds of random inputs, and if one fails it shrinks it to the simplest failing case.

from hypothesis import given, strategies as st

@given(st.lists(st.integers(), min_size=1))
def test_median_matches_statistics(xs):
    assert median(xs) == statistics.median(xs)

The lab's three functions all pass their example tests. Run the property tests and see how fast random inputs expose them.

In Python, a function is an object. You can store it in a variable, put it in a list, pass it to sorted(words, key=len), or return it from another function. A function that takes or returns functions is a higher-order function.

A decorator is a higher-order function used with @. It takes your function and returns a new one that wraps it: logging each call, timing it, caching results, checking permissions, or retrying on failure. The @ line is shorthand for f = decorator(f).

import functools

def log(func):
    @functools.wraps(func)
    def wrapper(*args):
        print(f"calling {func.__name__}{args}")
        result = func(*args)
        print(f"returned {result}")
        return result
    return wrapper

@log
@functools.cache
def slow_square(n): ...

Stacked decorators apply from the bottom up, so the one written on top is the outermost wrapper and runs first. The lab shows why order matters: put @log outside @cache and every call is logged; put it inside and only the cache misses are.

The function that notices a problem is rarely the one that knows how to handle it. A parser deep inside your program finds a bad line, but only the top-level code knows whether to skip the file, ask the user, or give up.

Exceptions connect the two. raise ValueError("bad line 3") stops the current function immediately. Python then unwinds the stack: it leaves each calling function in turn, running its finally blocks, until it reaches a try block with a matching except. If nothing catches it, the program stops and prints a traceback listing every frame it passed through.

try:
    config = load_config("app.toml")
except ValueError as err:
    print(f"Using defaults: {err}")
    config = DEFAULTS
finally:
    log.flush()          # runs no matter what

Python leans on exceptions more than most languages. Its style is called EAFP, "easier to ask forgiveness than permission": try the operation and handle the failure, instead of checking every precondition first. The with statement packages up the most common finally, closing files and releasing locks.

A program that downloads 4 web pages one after another spends most of its time waiting for the network. Concurrency means starting the next download while the first is still waiting, so the waits overlap.

Python offers three tools. Threads (threading, concurrent.futures) let the operating system switch between tasks. In standard CPython, though, a lock called the GIL (global interpreter lock) allows only one thread to run Python code at a time, so threads speed up waiting but not computing. asyncio runs many tasks in one thread. Each task says await wherever it would wait, and the event loop runs other tasks in the meantime. Processes (multiprocessing) each get their own interpreter and their own GIL, so they compute in parallel, at the cost of startup time and copying data between them.

import asyncio

async def fetch(url):
    await asyncio.sleep(1)      # stands in for network I/O
    return url

async def main():
    return await asyncio.gather(*(fetch(u) for u in urls))