Recursive Language Models Systems Lab

1.1 Threat Model for REPL-Based Agents

Goal

Shift your mental model from "can this work?" to "what can go wrong, and where?"

Threat Surfaces

• The model can emit unsafe code.
• The model can loop too long or recurse too aggressively.
• The execution environment can expose secrets, files, or network access.
• The trace can become too noisy to diagnose failures.
• Humans can misread a confident but incorrect chain of actions as evidence of correctness.

Minimum Threat Model

Describe each risk with three fields:

1. Trigger
2. Blast radius
3. Mitigation

For an RLM system, that means being specific about what happens when the model emits code, when that code runs, and what state it can observe or mutate.

Exercise

Take your toy harness and write one sentence each for:

• a confidentiality risk
• an integrity risk
• a cost risk
• an observability risk

1.2 Sandboxing and Step Budgets

Objective

Add two controls before you add more intelligence: a policy gate and a step budget.

Why These Two First

A recursive system fails in two boring ways before it fails in interesting ones:

• it runs code it should never have been allowed to run
• it keeps reasoning after the marginal value of another step has collapsed

Both failures are easier to prevent than to recover from.

Design Pattern

• Parse or inspect proposed code before execution.
• Maintain an explicit step_count.
• Stop on a hard limit, not on vibes.
• Log why the loop stopped.

Lab

Use the playground to implement a simple AST-based allowlist gate. Then write down where that gate is still insufficient for production.

1.3 Control Plane Checklist

Control Plane Checklist

If you had to explain your runtime to another engineer in one screen, what knobs would you show them?

Baseline Controls

• maximum recursive depth
• maximum total steps
• per-step timeout
• executor policy version
• allowed tools
• allowed filesystem scope
• network policy
• trace retention
• cost ceiling

Checkpoint

Write a short checklist for your own harness with:

1. Which control exists now
2. Which control is missing
3. Which missing control is the next highest leverage

2.1 Benchmark Design for Long-Context Workloads

Objective

Build evaluation tasks that actually stress the reason you would choose an RLM.

Bad Benchmarks

Bad benchmarks reward generic summarization, short-context guessing, or superficial retrieval. They do not force deliberate inspection.

Better Benchmark Families

• multi-hop extraction across distant sections
• codebase questions that require walking multiple files
• contradiction detection in long documents
• timeline reconstruction from scattered evidence
• constrained synthesis where intermediate state matters

Exercise

Draft three benchmark tasks of your own. For each one, write:

• what makes the context hard
• why naive prompting is likely to fail
• what evidence would tell you the RLM actually helped

2.2 Failure Taxonomy and Trace Review

Why Taxonomy Matters

If every miss is labeled "hallucination," you do not have an evaluation program. You have a complaint.

A Practical Taxonomy

• omission: the system failed to surface relevant evidence
• wrong synthesis: the evidence was found but combined incorrectly
• premature stop: the system answered before enough inspection
• over-recursion: the system kept going without improving the answer
• unsafe action: the model attempted a prohibited operation
• trace opacity: the logs do not explain the behavior well enough to debug it

Review Exercise

Take one failed run and label it using exactly one primary failure mode and one contributing factor. That discipline forces sharper fixes.

2.3 Instrumentation Lab

Lab Prompt

Instrument your toy harness so every step records:

• step number
• model action type
• code emitted or summary emitted
• execution result
• elapsed time
• stop reason

Minimum Output Shape

{
  "step": 4,
  "action": "python",
  "elapsed_ms": 182,
  "stop_reason": null
}

Reflection

After adding instrumentation, ask:

• What new failure did the logs reveal?
• What noise can be removed?
• What metric would you put on a dashboard first?

3.1 RLM vs RAG vs Long-Context Memo

Objective

Choose an architecture because it matches the workload, not because it is the newest or most intellectually satisfying.

Decision Heuristic

• Use long-context prompting when the task is simple and the context is manageable.
• Use RAG when lookup quality matters more than recursive reasoning.
• Use an RLM when the model must inspect, decompose, store intermediate state, and revisit different parts of the context deliberately.

Memo Exercise

Write a one-page memo for a concrete workload you care about. End with:

• recommended pattern
• rejected alternatives
• one assumption that would change the decision

3.2 Cost, Latency, and Quality Tradeoffs

Tradeoff Frame

Do not ask whether RLM is "better." Ask what you are buying with the extra machinery.

Typical Tradeoffs

• better inspection quality on hard tasks
• higher orchestration complexity
• more moving parts to secure and observe
• potentially lower prompt bloat
• potentially higher wall-clock latency if recursion is uncontrolled

Exercise

Estimate three numbers for one workload:

1. a plain prompt baseline
2. a RAG baseline
3. an RLM design

Use rough numbers. The goal is comparative reasoning, not fake precision.

3.3 Capstone Design Review

Capstone

Pick one serious target workload:

• large codebase analysis
• long-document review
• multimodal workflow with many intermediate artifacts
• agentic computer-use trace analysis

Review Questions

• Why is the workload hard?
• What does the model need to inspect, and what can stay external?
• What controls are mandatory before production?
• What benchmark suite would you use?
• What would make you choose RAG or long-context prompting instead?

Deliverable

Produce a short design review with five sections:

1. Problem
2. Runtime design
3. Safety controls
4. Evaluation plan
5. Go / no-go recommendation