Higher-order functions and composable behavior

Prerequisites: Functions, return values, lists, and variable scope.

Learning objectives

• Write functions that accept or return functions.
• Trace a closure and its captured environment.
• Distinguish pure transformations from effects.

A function can be an argument

Higher-order functions take functions as inputs or return them as results. This lets one traversal work with several rules. A filter operation needs a collection and a predicate deciding which elements to retain. A map operation needs a transformation. Neither operation needs to know the application-specific meaning of the values.

Composition connects outputs to inputs. If f consumes a value and g consumes f's result, their composition computes g(f(x)). Order matters unless you prove otherwise. Rounding a number before multiplying can produce a different result from multiplying before rounding. The types and contracts must also agree at the connection.

def compose(outer, inner):
    def combined(value):
        return outer(inner(value))
    return combined

def add_three(value):
    return value + 3

def double(value):
    return value * 2

pipeline = compose(double, add_three)
assert pipeline(4) == 14

Calling compose does not immediately call add_three or double. It creates and returns combined. That returned function retains access to the outer call's bindings for outer and inner. A function together with the environment needed to resolve its free names is a closure.

Captured state can still change

A closure is not necessarily a frozen photograph of every value it can reach. If it captures a mutable list, later mutations can affect its behavior. In Python, closures can also refer to enclosing bindings whose values change. Predict behavior by tracking the environment and referenced objects, not by assuming creation time fixes every future result.

Pure functions make composition easier to reason about: their result depends on their arguments and they do not modify externally visible state. Replacing a call with its result preserves observable behavior under those assumptions. A function that prints, reads a clock, or updates a shared counter has effects, so changing evaluation order or calling it twice can change the program.

Higher-order abstractions are useful when they remove repetition while preserving clarity. A named function can communicate a domain rule better than a long anonymous expression. Error handling also composes only when specified: if inner raises an exception, outer is never called in this implementation. If transformations may return missing results, the pipeline must define how that condition propagates. Treat behavior passed as data with the same contract discipline used for ordinary data, including assumptions, effects, and failure modes.

Practice problem

Design make_between(low, high) to return a predicate accepting numbers in the inclusive interval. Use two independently created predicates for [0,10] and [20,30], and show that their bounds do not overwrite one another. Combine one predicate with a separate even-number predicate to select values from [-2,0,3,8,10,12]. Specify what should happen when low is greater than high. Explain how the design changes if the bounds are intentionally stored in a mutable configuration object that can change after predicate creation.

Worked solution

Create a nested function that returns low <= value <= high and return that function. Each call to make_between creates a separate enclosing environment, so predicates from distinct calls retain distinct bound bindings. The [0,10] predicate combined with evenness retains [0,8,10]. Reject reversed bounds at construction with a documented exception, or explicitly define an empty interval; either is coherent if callers know the policy. Capturing a mutable configuration object means calls can observe later bound changes, which may be desired for a live policy but differs from a fixed predicate. Tests should construct both predicates, call them in alternating order, check exact endpoints, and, if mutable configuration is supported, verify the specified update behavior.

Check your understanding

In compose(outer, inner), when combined(x) runs, which function receives x first?

• outer.
• inner.
• Neither; both receive the final result.

Explanation

The body evaluates inner(value) before using its result as the argument to outer.

A closure captures a list and later the list is mutated. Which statement is generally correct?

• The closure must still see the old contents.
• The closure becomes invalid.
• The closure can observe the changed contents through the shared reference.

Explanation

Capturing an environment does not automatically deep-copy mutable objects.

Primary reading

• Structure and Interpretation of Computer Programs: University of California, Berkeley
• The Python Tutorial: Python Software Foundation

Open the study and save your work →