递归语言模型运行时

Install dependencies

uv sync

Installation

pip install pyrlm-runtime

Or with uv:

uv add pyrlm-runtime

For live terminal visualization of the REPL loop with rich:

pip install "pyrlm-runtime[rich]"

Requirements: Python 3.12+

Optional: For the secure Monty REPL backend (Rust sandbox):

pip install pydantic-monty

Requires: pip install pydantic-monty

rlm = RLM(adapter=adapter, repl_backend="monty") ```

How MontyREPL handles complex objects: Python objects like Context can't run natively in the Rust sandbox. MontyREPL uses an object proxy system — methods are registered as external functions with {name}__{method} naming, and AST rewrites transform ctx.method() calls into ctx__method() calls transparently.

Variable persistence: MontyREPL uses AST-based detection of assignments, appending a capture dict to extract variable state from each execution.

Both backends implement the same REPLProtocol interface: exec(code) -> ExecResult, get(name), set(name, value).

Quick Setup

First, install the optional Elasticsearch extra:

pip install "pyrlm-runtime[elasticsearch]"

from pyrlm_runtime import RLM
from pyrlm_runtime.adapters import OpenAICompatAdapter
from pyrlm_runtime.retrieval import ElasticsearchRetriever

retriever = ElasticsearchRetriever(
    host="https://my-cluster.es.cloud.com",
    api_key="xxx",
    index="pdf_corpus",
    embedding_model="text-embedding-3-small",
)

rlm = RLM(adapter=OpenAICompatAdapter(model="gpt-5"), retriever=retriever)
answer, trace = rlm.run("Who signed document X?")  # No context needed

When a retriever is configured, four functions become available in the REPL:

es_search(query, top_k=10, filters=None)        # BM25 keyword search
es_vector_search(query, top_k=10, filters=None)  # Semantic similarity
es_hybrid_search(query, top_k=10, filters=None)  # Combined (recommended)
es_get(doc_id)                                    # Fetch full document

The retrieval layer is backend-agnostic: any object implementing the RetrieverProtocol (with search, vector_search, hybrid_search, get methods) works as a drop-in backend.

Quickstart

2. Basic usage

```python from pyrlm_runtime import RLM, Context from pyrlm_runtime.adapters import OpenAICompatAdapter

Common configurations by use case

Examples

Run any example:

uv run python examples/minimal.py

Quick demo

RLM_CONTEXT_SIZES=5,30 uv run python examples/rlm_vs_baseline.py

Optional: custom endpoint (Ollama, LM Studio, etc.)

Retrieval (when retriever is configured)

es_search(query, top_k=10, filters=None)
    # BM25 full-text search → list of {doc_id, preview, score, metadata}

es_vector_search(query, top_k=10, filters=None)
    # Semantic similarity search → list of {doc_id, preview, score, metadata}

es_hybrid_search(query, top_k=10, filters=None)
    # Combined BM25 + semantic (recommended) → list of {doc_id, preview, score, metadata}

es_get(doc_id)
    # Fetch full document → {doc_id, content, metadata}

Configuration

Environment variables

```bash

Custom endpoint (optional)

LLM_BASE_URL="https://..."

1. Set your API key

```bash export LLM_API_KEY="your-api-key-here"

3. Run without external APIs (for testing)

from pyrlm_runtime import RLM, Context
from pyrlm_runtime.adapters import FakeAdapter

adapter = FakeAdapter(script=[
    "snippet = peek(80)\nsummary = llm_query(f'Summarize: {snippet}')\nanswer = f'Summary -> {summary}'",
    "FINAL_VAR: answer",
])
adapter.add_rule("You are a sub-LLM", "[fake] short summary")

context = Context.from_text("RLMs treat long prompts as environment state.")
output, trace = RLM(adapter=adapter).run("Summarize this.", context)
print(output)  # Summary -> [fake] short summary

API Reference

Any OpenAI-compatible endpoint

adapter = OpenAICompatAdapter( model="my-model", base_url="https://my-endpoint.com/v1", )

Uses environment variables: `LLM_API_KEY` (or `OPENAI_API_KEY`), `LLM_BASE_URL`.

#### GenericChatAdapter

For non-standard APIs with custom request/response formats.

adapter = GenericChatAdapter( base_url="https://custom-api.com", path="/chat/completions", model="custom-model", api_key="your-key", payload_builder=my_custom_builder, # Custom request format response_parser=my_custom_parser, # Custom response format timeout=60.0, max_retries=3, )

Auto-retries on 429, 500, 502, 503, 504 with exponential backoff. Supports context manager (`with GenericChatAdapter(...) as adapter:`).

#### FakeAdapter

Deterministic adapter for testing. No external API needed.

adapter = FakeAdapter( script=["code step 1", "code step 2", "FINAL_VAR: result"] )

API key (checked in order)

LLM_API_KEY="your-key" # Primary OPENAI_API_KEY="your-key" # Fallback

References

• MIT CSAIL Paper: Recursive Language Models — Zhou, et al.
• This implementation is not affiliated with MIT.

Retrieval Integration

For large corpora that don't fit in memory, the RLM can search external document indexes directly from the REPL loop. See the detailed architecture guide: docs/RETRIEVAL.md

Benchmark: RLM vs Baseline

The rlm_vs_baseline.py example reproduces the key finding from the MIT paper (Figure 1): RLMs maintain accuracy as context grows, while baseline approaches degrade due to truncation.

Figure 1: RLM accuracy remains high as distractor documents increase, while baseline accuracy drops.

Ask questions over the entire context

answer, trace = rlm.run("What are the main themes across all documents?", context) print(answer) ```