redef¶

A correctness kit for decorator-scoped AST rewriting in Python.

redef is for library authors who transform a decorated function's code: recover its source, rewrite the AST, and re-define the function from the result. That pipeline looks like four lines with inspect, ast, and compile, and every deployed implementation of it we audited gets a different subset of the correctness details wrong. redef packages the pipeline with the details on by default.

import ast
from redef import rewrite

class Explain(ast.NodeTransformer):
    """Rewrite `assert cond` into a check that reports its operands."""
    ...

@rewrite(Explain())          # innermost decorator, always
def total(xs):
    s = 0
    for x in xs:
        s += x
    assert s >= 0
    return s

The decorated file stays ordinary Python: no import hook, no source codec, no global state, and every static checker reads it unchanged. Only decorated functions pay, at tens to hundreds of microseconds each, once, at definition time.

What "the details" means¶

Seven things, each measured or found in the field before the library existed:

1. Dedent recovered source, or nested functions fail to parse.
2. Strip the decorator list before recompiling, or the rewrite re-applies itself.
3. Offset line numbers by the function's position in its file, or every traceback points near line 2.
4. Verify the recovered source defines your function. A stale linecache entry (a re-executed notebook cell) hands you valid source for the wrong function, and the naive pipeline rewrites it silently.
5. Refuse closures at recompile time, loudly, instead of letting free variables silently become global lookups that fail at call time.
6. Keep original default objects. The recompiled def re-evaluates default expressions, breaking identity for mutable defaults.
7. Inherit the module's __future__ flags. They live in co_flags, appear nowhere in the function's text, and change what your recompile means.

Beyond the repairs, redef contributes verify-by-recompile: it recompiles the recovered, untransformed source and compares code objects structurally, catching the one stale-source case a name check cannot see, a stale body under the same name. The check is on by default and measured: zero false positives across 330 cells (22 function shapes, five CPython versions, three optimization levels), plus a field trial on a real 73-test suite. See Verification.

Where it works, stated up front¶

Source recovery has a fixed envelope: 10 of 18 execution contexts on CPython 3.14, and no library can widen it. What redef does at the edges is fail with the context named and the remedy stated, instead of misbehaving. A frozen app raises SourceUnavailable("frozen-or-sourceless"), and code under exec raises with the two-line recipe that makes it recoverable. The full matrix is in Support matrix.

Status¶

Unreleased research software, developed inside the cs-lab research programme that measured the gap it fills. The API is small and stable in shape; the name, packaging, and versioning are not. Its test suite is the research programme's experiment battery.