FREE COMPUTER SCIENCE STACK LEVEL · 11 OF 20

Languages & compilers

How do syntax and semantics become machine behavior?

The question this layer answers

How do syntax and semantics become machine behavior?

Human intent becomes execution. This layer connects Parsers, Types, Compilers, VMs, Garbage collection to the rest of the computing stack.

Syntax is structure, not a sequence of characters

Consider 2 + 3 * 4. We want the multiplication to bind more tightly than addition. A lexer can identify number and operator tokens, but a parser must construct the grouping. One suitable grammar is:

expression := term ("+" term)*
term       := factor ("*" factor)*
factor     := NUMBER | "(" expression ")"

The grammar makes a term the unit that addition combines. Multiplication is resolved inside each term. The resulting abstract syntax tree is Add(Number(2), Multiply(Number(3), Number(4))). Parentheses in (2 + 3) * 4 produce a different tree, and therefore a different result.

Define meaning recursively

An evaluator follows the tree:

evaluate(Number(n))       = n
evaluate(Add(left,right)) = evaluate(left) + evaluate(right)
evaluate(Multiply(l,r))   = evaluate(l) * evaluate(r)

Evaluating the first tree computes 3 × 4 = 12, then 2 + 12 = 14. The parenthesized tree computes 2 + 3 = 5, then 5 × 4 = 20. The recursive structure of evaluation mirrors the recursive structure of the syntax.

Add variables and you need an environment: a mapping from names to values. Looking up an absent name should produce a defined error rather than silently guessing a value. Add functions and you must specify how environments are created and captured. Add mutation and evaluation order can become observable. Each feature expands the semantic contract.

Separate error stages

2 + * 3 contains recognizable tokens but violates the grammar. unknown + 3 can parse correctly while failing name lookup. "hello" + 3 may be rejected by a type checker or interpreted according to an explicitly defined coercion rule. These are not interchangeable “syntax errors.” Distinguishing stages makes tools more useful and programs easier to debug.

A compiler need not execute the tree immediately. It can translate the tree into an intermediate representation and then machine instructions. An optimization is valid only when it preserves the language's observable behavior under stated assumptions. Replacing x * 0 with zero is straightforward for mathematical integers, but deserves more care if evaluating x has side effects or if floating-point exceptional values matter.

Build the smallest honest language

Implement just numbers, addition, multiplication, and parentheses before adding variables. Test valid nested expressions, extra closing parentheses, missing operands, whitespace, and unexpected characters. Require the parser to consume the entire input: accepting a valid prefix while ignoring trailing junk can hide an error.

Worked checkpoint: write the tree for 2 * (3 + 4) + 5 before calculating. It is Add(Multiply(2, Add(3,4)),5), evaluating to 19. The lab asks you to explain that structure, not merely guess the arithmetic result.

Free Languages & compilers lessons

1. Source becomes structure

Lexing, parsing, grammars, and syntax trees

Lexing commonly groups characters into tokens; parsing checks those tokens against a grammar and constructs a syntax tree. Precedence and associativity decide whether a+b*c means a+(b*c). The tree discards punctuation that no longer matters while preserving the nesting required for later meaning.

A grammar that admits two parse trees for one token sequence leaves meaning underdetermined unless another rule resolves it. Languages define precedence, associativity, or explicit delimiters; parser generators may report conflicts. Silently accepting whichever parse happens first is not a sound semantics.

2. Types make impossible states harder to express

Static constraints and runtime meaning

Static checking rejects some programs before execution; dynamic checking attaches decisions to runtime values. Rich types can encode variants, ownership, nullability, effects, or units. No practical type system proves every desirable property, and unsafe escape hatches or external data reintroduce obligations.

Instead of a nullable result with side-channel error codes, define Success(value) or Failure(reason). Pattern matching then requires each variant to be handled, and the payload appropriate to one variant is unavailable in the other. The representation moves a runtime convention into a checkable structure.

3. Many roads to execution

Compilers, interpreters, VMs, and JITs

A native compiler can optimize ahead of time into target instructions. An interpreter can execute a syntax tree or bytecode step by step. A virtual machine defines a portable instruction set. A just-in-time compiler observes running code and compiles hot paths using runtime facts, with warm-up and deoptimization costs.

Constant folding can replace 3×7 with 21; dead-code elimination can remove work whose result is unobservable; inlining exposes cross-function opportunities. But exceptions, I/O, timing contracts, concurrency, and language rules constrain what counts as unobservable. Faster wrong code is not an optimization.

4. Language mastery: who owns memory?

Lifetime, allocation, garbage collection, and boundaries

Manual management gives explicit control but risks leaks, use-after-free, and double release. Tracing garbage collection discovers objects reachable from roots and reclaims the rest, trading control for runtime work and pauses. Ownership systems encode lifetime constraints statically. Reference counting is prompt but struggles with cycles unless supplemented.

Managed code calling native code must agree on layout, ownership, error signalling, threading, and how objects remain alive while native pointers exist. A type-safe caller can still be corrupted by a callee that writes beyond a buffer. Safe wrappers minimize and validate the unsafe surface.

Practice lab: Give an expression a structure

About 30 minutes. Know arithmetic, parentheses, and the difference between an instruction and the value it computes.

Your tiny language has integer literals, +, *, and parentheses. Multiplication binds more tightly than addition; both operators associate left. There are no side effects or overflow. Parse 2 + 3 * 4 and (2 + 3) * 4. Use Number(n), Add(left,right), and Multiply(left,right) as tree nodes.

  1. List the tokens and draw both trees. Evaluate from the leaves upward.
  2. Write pseudocode for evaluate(node) with one case per node type. What should an unknown node do?
  3. Add an invalid expression, 2 + * 4. Explain why tokenization can succeed while parsing fails.
Hint

The outermost operation is the root of the tree, not necessarily the first operator in the source.

Worked solution

The first tree is Add(Number(2), Multiply(Number(3), Number(4))) and evaluates to 14. The second is Multiply(Add(Number(2), Number(3)), Number(4)) and evaluates to 20. evaluate(Number(n)) returns n; evaluate(Add(l,r)) returns evaluate(l) + evaluate(r); Multiply uses multiplication. Unknown nodes report an error. The tokens [2,+,*,4] are individually legal, but the grammar requires an operand after + and finds * instead. Parsing decides structure; evaluation applies meaning. Constant folding can replace either complete tree with its numeric result only under our side-effect-free, no-overflow assumptions.

  • Draw distinct trees and get 14 versus 20.
  • Cover every node case and invalid input.
  • Separate lexical validity, grammar, and evaluation.

Change one assumption

If + can concatenate strings as well as add integers, define what 2 + "3" means. Explain why the parser alone cannot decide that policy.

Connect syntax, meaning, and implementation

Build small derivations and counterexamples. Parsing, typing, storage management, and optimization each preserve a different part of a language contract.

1. Parse and evaluate · allow 40 minutes

A grammar must choose a tree

Use the grammar E→T {+T}, T→U {U}, U→−U | number | (E). Braces mean repetition and operators in each repetition associate left. Tokenize and draw the abstract syntax tree for −23+4 and for 2+3*4. Evaluate both.

Explain why E→E+E | EE | number is ambiguous for 2+34 by drawing two trees with different results. Extend the original grammar with left-associative subtraction at the addition level, then parse 8−3−2. State which errors belong to lexing and which to parsing.

Get a starting hint

A grammar can encode precedence through separate nonterminals. Associativity determines how repeated operators group at one precedence level.

Compare your reasoning with the worked solution

The first tree is Add(Mul(Neg(2),3),4), evaluating to−2. The second is Add(2,Mul(3,4)), evaluating to 14. The ambiguous grammar also permits Mul(Add(2,3),4), evaluating to 20, so the same token sequence has two meanings unless an extra disambiguation rule is supplied.

Extend E to T {( + | − ) T}. Folding the repetition left gives Sub(Sub(8,3),2)=3. A right fold would give Sub(8,Sub(3,2))=7 and violate the stated associativity. Unary minus remains in U, so it binds more tightly than multiplication in this toy language.

A lexer recognizes tokens and can reject an unsupported character or malformed numeral under its lexical rules. A parser can reject a valid token sequence such as 2+*3 or an unmatched parenthesis because it has no grammatical structure. Neither phase establishes all semantic properties, such as whether a referenced name exists or a division will use zero.

Check your reasoning

  • Draw trees that make precedence and associativity visible.
  • Give two interpretations to demonstrate ambiguity.
  • Distinguish lexical, grammatical, and semantic rejection.

2. Environment model · allow 35 minutes

Trace shared and independent captured state

A counter factory creates local n=0 and returns an increment function that mutates that same captured n and returns it. Let c=make_counter(), d=c, and e=make_counter(). Predict c(),d(),e(),c() and draw environments and function references.

Contrast this with a factory returning a function that adds an immutable captured argument x to its input. Explain why changing an unrelated outer variable also named x does not change the captured binding in a lexically scoped language. Identify the lifetime problem an implementation must solve when a closure outlives its factory's call frame.

Get a starting hint

d and c name the same closure; e comes from a different factory invocation. Lexical scope follows definition environments, not spelling alone.

Compare your reasoning with the worked solution

The results are 1,2,1,3. c and d reference one closure with one mutable cell for n; e references a separate closure with a separate cell. Each factory call creates its own binding even though both variables are named n in the source. Assigning d=c copies a function reference, not the captured environment.

For make_adder(x), the returned function resolves x through the environment captured at its definition. An unrelated caller variable named x does not replace that binding. If the captured binding itself is mutable and the language permits mutation, later changes to that binding can be observed; lexical scope is not a universal promise of value snapshots.

A closure that survives the call cannot safely retain a pointer into a stack slot that will be reused. Implementations can move captured cells to longer-lived storage, close upvalues when a frame exits, or use another representation preserving lifetime and sharing semantics. Copying the value separately into c and d would be wrong for the mutable-counter contract because updates must remain shared.

Check your reasoning

  • Distinguish closure identity from a second factory invocation.
  • Explain binding capture without assuming all captures are immutable snapshots.
  • Preserve both lifetime and intentional sharing.

3. Typing derivation · allow 40 minutes

A type judgment proves a specific property

In a toy language, integers and booleans are distinct types. Addition requires two integers. An if expression requires a boolean condition and both branches of the same type. Analyze: let x=3 in if true then x+1 else false. Give the successful and failed typing premises.

Repair it in two ways while keeping the language rules unchanged. Explain why its untyped evaluator might still return 4. Then analyze 1/0 when division accepts integer operands: which check is satisfied, which failure remains, and what extra specification would be needed to exclude it statically?

Get a starting hint

Typing rules describe a language's chosen proof discipline. Reachability and ordinary type compatibility are different analyses.

Compare your reasoning with the worked solution

Under x:Int, x+1 has type Int because both operands do. The condition true has type Bool. The else branch false has type Bool, so the branch-equality premise fails and the if expression has no type under the stated rules. Replace false with an integer such as 0, or replace the then branch x+1 with the boolean literal true. Both repairs use only the stated types and rules.

An untyped evaluator that evaluates only the selected branch returns 4 without visiting false. That execution does not establish that the program satisfies this static typing discipline. A language could include more precise reachability reasoning, but doing so changes the analysis rather than proving the existing judgment succeeds.

For 1/0, both operands satisfy the integer-type requirement. The zero divisor violates a separate operational precondition. Runtime checks, a nonzero-divisor type with suitable construction rules, or a refinement analysis could address it. A claim of type safety must specify which errors are excluded and which remain permitted behaviors or exceptions.

Check your reasoning

  • Locate the exact failed rule premise.
  • Provide repairs that satisfy the declared rules.
  • Avoid treating ordinary type correctness as proof of all runtime safety.

4. Intermediate representation · allow 40 minutes

Merge control flow without inventing a constant

Translate this code into a small SSA-style form: if c then x=2 else x=5; y=x*3; return y. Give distinct names for branch assignments and a merge operation that chooses according to the predecessor block.

List possible returned values when c is unknown, then propagate constants when c is known true. Explain why replacing the merge with either input is unsound when c is unknown. Now suppose the untaken branch contains a call that prints a message. State when removing that call preserves observable behavior and when moving it before the branch does not.

Get a starting hint

A phi-like merge selects the value from the executed predecessor; it is not a runtime choice independent of control flow.

Compare your reasoning with the worked solution

The true block defines x1=2 and the false block x2=5. At the join, x3=phi(trueBlock:x1,falseBlock:x2); then y1=x3*3; return y1. With unknown c, returns are 6 or 15. If c is proven true for the relevant execution, the false edge is unreachable, x3 becomes 2, and y1 becomes 6.

Choosing x1 unconditionally would return 6 even when c is false; choosing x2 has the symmetric error. SSA names encode definitions, while the merge preserves which definition reaches the use. A conservative value analysis can record the set{2,5} without pretending it is a single constant.

A call in a branch proven unreachable under the language's evaluation rules can be removed because it would not execute. Hoisting a printing call before a branch makes it execute on paths where it previously did not, changing observable output. Similar concerns apply to exceptions, volatile access, and memory mutation. Optimization needs a definition of observable behavior and a proof relative to it.

Check your reasoning

  • Associate each merge input with a predecessor.
  • Propagate only facts justified by control-flow information.
  • Account for observable effects when moving or deleting operations.

5. Reachability and mutation · allow 45 minutes

Collect a cycle and preserve a newly reachable object

Objects have edges A→B,B→C,C→B,D→E,E→D, with F having no outgoing edge. Roots are A and F. Trace a mark phase and identify collectible objects. Compute incoming reference counts including root references and explain why simple reference counting does not collect every unreachable object here.

In a separate incremental-collection example, A is black (already scanned), G is gray (reachable but not yet scanned), and G points to white D. Before G is scanned, the mutator moves that reference: it adds A→D and removes G→D. Explain how D could be collected incorrectly without a write barrier, and describe one barrier strategy that prevents this specific error.

Get a starting hint

Reachability starts at roots. A positive incoming count can come entirely from an unreachable cycle.

Compare your reasoning with the worked solution

Tracing marks A and F from roots, then B from A and C from B. The C→B edge reaches an already marked object. D and E remain unmarked and can be collected. Counts are A: 1 from its root; F: 1 from its root; B: 2 from A and C; C: 1 from B; D: 1 from E; E: 1 from D. The unreachable D/E cycle has no zero-count object, so simple immediate reference counting leaves it allocated.

In the separate incremental example, moving the reference from gray G to black A keeps D reachable, but the later scan of G no longer discovers it and A is not automatically rescanned. D may therefore remain white. Sweeping white objects would then reclaim live D. An insertion barrier can shade D when a scanned object gains a reference to it, ensuring D is scheduled for tracing; another strategy can record and rescan the affected source. Exact barriers depend on the collector's invariant.

The example separates liveness from allocation age and reference count. A moving collector adds another obligation: every relevant reference and root must be updated or mediated through a representation that remains valid after relocation.

Check your reasoning

  • Trace from roots rather than count all edges as liveness evidence.
  • Identify the unreachable cycle explicitly.
  • State the collector invariant protected by the barrier.

6. Counterexamples and proof obligations · allow 40 minutes

Find the semantic contract behind an algebraic rewrite

Evaluate the rewrite x+1>x → true in three toy arithmetic models: unbounded integers, signed 32-bit wraparound, and signed 32-bit checked addition that traps on overflow. Give a boundary input for the latter two.

Evaluate x0 → 0 for IEEE-style floating-point values that may include infinity or NaN. Finally consider f()+f() → 2f() when f increments and returns a shared counter. Give a concrete initial state and compare final states and values. State assumptions that would make each proposed optimization legitimate.

Get a starting hint

Observable behavior includes exceptions and state changes, not only the final number on ordinary inputs.

Compare your reasoning with the worked solution

For unbounded integers x+1>x always holds. In 32-bit signed wraparound, x=2^31−1 produces−2^31 after adding 1, so the comparison is false. In checked arithmetic, that input traps instead of returning a boolean. Replacing either with true changes behavior. A proven range excluding overflow can justify the rewrite for the bounded models.

For floating point, infinity times zero and NaN times zero yield NaN under the usual IEEE operations, not numeric zero. Signed-zero distinctions or floating-point exception behavior can add further obligations. A restricted finite-value contract and permitted treatment of zero signs/exceptions must be explicit.

Let the counter start 0 and f increment before returning. f()+f() yields 1+2=3 and leaves counter 2. 2*f() yields 2 and leaves counter 1. Purity and repeatable return values would be needed to remove one call, and arithmetic overflow semantics still need checking. The general method is to list the language's observations, then prove the transformation preserves them for every permitted input.

Check your reasoning

  • Use boundary inputs permitted by each arithmetic model.
  • Include traps and state changes in observable behavior.
  • State sufficient preconditions rather than dismiss every optimization.

Study the underlying methods

Section project: Implement a small expression language end to end

6 to 8 hours. Build an interpreter for a deliberately small language containing integer literals, Boolean literals, variables, addition, multiplication, equality, if expressions, and lexical let bindings. Inputs are source strings; outputs are a token stream, a syntax tree, a type result, and either an evaluated value or a diagnostic with source position. Publish an unambiguous grammar with multiplication binding more tightly than addition and explicit parentheses. Give integers and Booleans distinct types even if the host language treats Booleans as a kind of integer. A let binding is visible only in its body; evaluating a nested binding must not overwrite an outer environment. The type checker rejects arithmetic on Booleans and requires both if branches to have the same type. The evaluator executes only the selected branch, although the type checker inspects both. Implement each stage with ordinary Python data structures and no parsing package. Submit examples that expose tokenization, precedence, scope, static rejection, and runtime control flow rather than only examples that happen to produce a number. Give a source example rejected at each stage and explain why later stages are not run after an earlier failure. Distinguish a syntactically valid but ill-typed program from a malformed expression.

Milestones

  1. Write the grammar, token definitions, value types, and evaluation rules.
  2. Implement a lexer and precedence-aware parser with position-aware errors.
  3. Implement lexical environments and a separate recursive type checker.
  4. Implement evaluation and a command-line interface displaying each stage.
  5. Create boundary examples and compare expected trees, types, and values.

Acceptance criteria

  • The expression 2+3*4 produces 14, while (2+3)*4 produces 20, with different correct syntax trees.
  • A nested let binding shadows its outer name only inside its own body.
  • Arithmetic involving true is rejected by the language type checker rather than silently using Python integer behavior.
  • An if expression with differently typed branches is rejected even when its condition is a literal.
  • An unbound variable and an unexpected token report their source locations and distinct diagnostic categories.
  • Evaluation visits only the selected if branch, demonstrated by an instrumented evaluation trace.

Add first-class functions and closures, specifying lexical capture and a type policy before implementing application.

Practise Languages & compilers free →