# RAG MCP Server Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build a RAG MCP server that lets Engineer, Architect, QAPlanner, and QAEngineer agents search a pgvector knowledge base (codebase, memory, docs) via on-demand tool calls.

**Architecture:** A standalone FastMCP server (`rag-mcp/`) exposes `search_codebase`, `search_memory`, and `search_docs` tools over the Streamable HTTP transport. Embeddings are generated by a configurable backend (Ollama/vLLM/OpenAI). Vectors are stored in pgvector on the existing Postgres. An `indexer.py` CLI populates the knowledge base. Agent integration adds `tool_registry` support to three agents and wires up the orchestrator.

**Tech Stack:** Python 3.11+, mcp 1.27 (FastMCP), psycopg2-binary, pgvector, requests, uvicorn, Docker

---

## File Map

**New files (`rag-mcp/`):**
- `rag-mcp/models.py` — Pydantic models: `SearchResult`, `EmbedderError`
- `rag-mcp/db.py` — psycopg2 connection pool + `search_chunks()`, `upsert_chunk()`
- `rag-mcp/embedder.py` — `Embedder` class with Ollama/vLLM/OpenAI backends
- `rag-mcp/indexer.py` — CLI: chunk files + upsert into pgvector
- `rag-mcp/main.py` — FastMCP server with 3 tools and /health
- `rag-mcp/migrations/001_create_rag_embeddings.sql` — pgvector schema
- `rag-mcp/requirements.txt` — server-only Python deps
- `rag-mcp/Dockerfile` — container build
- `rag-mcp/docker-compose.yml` — standalone compose for the RAG service

**Modified files:**
- `agents/engineer.py` — add `tool_registry` param + `call_with_tools` path
- `agents/architect.py` — add `tool_registry` param + `call_with_tools` path
- `agents/qa_engineer.py` — add `tool_registry` param + `call_with_tools` path
- `orchestrator.py` — extract RAG server config, create `rag_registry`, pass to 3 agents
- `config.yaml` — add `mcp.servers` RAG entry example

---

### Task 1: Directory skeleton + SQL migration + models.py

**Files:**
- Create: `rag-mcp/migrations/001_create_rag_embeddings.sql`
- Create: `rag-mcp/models.py`
- Create: `rag-mcp/__init__.py`

- [ ] **Step 1: Create directory tree**

```bash
mkdir -p /home/wanleung/Projects/ai-software-house/rag-mcp/migrations
touch /home/wanleung/Projects/ai-software-house/rag-mcp/__init__.py
```

- [ ] **Step 2: Write the SQL migration**

Create `rag-mcp/migrations/001_create_rag_embeddings.sql`:

```sql
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS rag_embeddings (
    id          BIGSERIAL PRIMARY KEY,
    source_type TEXT NOT NULL,
    source_id   TEXT NOT NULL,
    chunk_index INT  NOT NULL,
    content     TEXT NOT NULL,
    embedding   vector(768),
    metadata    JSONB DEFAULT '{}'
);

CREATE INDEX IF NOT EXISTS rag_embeddings_hnsw_idx
    ON rag_embeddings USING hnsw (embedding vector_cosine_ops);

CREATE UNIQUE INDEX IF NOT EXISTS rag_embeddings_upsert_idx
    ON rag_embeddings (source_type, source_id, chunk_index);
```

- [ ] **Step 3: Write models.py**

Create `rag-mcp/models.py`:

```python
"""Pydantic models for the RAG MCP server."""
from __future__ import annotations

from typing import Any

from pydantic import BaseModel


class SearchResult(BaseModel):
    content: str
    source_id: str
    chunk_index: int
    score: float
    metadata: dict[str, Any] = {}


class EmbedderError(Exception):
    """Raised when the embedding backend is unreachable or returns an error."""
```

- [ ] **Step 4: Verify files exist**

```bash
ls rag-mcp/migrations/ && python -c "from rag_mcp.models import SearchResult, EmbedderError; print('OK')" 2>/dev/null || python rag-mcp/models.py && echo "models OK"
```

Expected: `001_create_rag_embeddings.sql` listed; no import errors.

- [ ] **Step 5: Commit**

```bash
git add rag-mcp/
git commit -m "feat(rag): scaffold directory, migration, and models"
```

---

### Task 2: db.py — pgvector connection and search helpers

**Files:**
- Create: `rag-mcp/db.py`

- [ ] **Step 1: Write the failing test**

Create `tests/test_rag_db.py`:

```python
"""Tests for rag-mcp/db.py — mocked psycopg2, no live DB required."""
import sys, os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "rag-mcp"))

import json
from unittest.mock import MagicMock, patch
import pytest


def make_mock_conn(rows=None):
    """Return a mock psycopg2 connection whose cursor returns rows."""
    rows = rows or []
    cursor = MagicMock()
    cursor.fetchall.return_value = rows
    conn = MagicMock()
    conn.cursor.return_value.__enter__ = lambda s: cursor
    conn.cursor.return_value.__exit__ = MagicMock(return_value=False)
    return conn, cursor


def test_search_chunks_returns_search_results():
    """search_chunks() maps pgvector rows to SearchResult objects."""
    from db import search_chunks
    from models import SearchResult

    row = ("hello world", "src/main.py", 0, 0.92, json.dumps({"lang": "python"}))
    conn, cursor = make_mock_conn(rows=[row])

    with patch("db._get_conn", return_value=conn):
        results = search_chunks("codebase", "database", top_k=3)

    assert len(results) == 1
    r = results[0]
    assert isinstance(r, SearchResult)
    assert r.content == "hello world"
    assert r.source_id == "src/main.py"
    assert r.chunk_index == 0
    assert abs(r.score - 0.92) < 0.001
    assert r.metadata == {"lang": "python"}


def test_search_chunks_filters_by_source_type():
    """search_chunks() passes source_type as a WHERE filter."""
    from db import search_chunks

    conn, cursor = make_mock_conn(rows=[])

    with patch("db._get_conn", return_value=conn):
        search_chunks("memory", "query", top_k=5)

    sql_called = cursor.execute.call_args[0][0]
    assert "source_type" in sql_called


def test_upsert_chunk_calls_execute_and_commit():
    """upsert_chunk() inserts or updates a chunk row."""
    from db import upsert_chunk

    conn, cursor = make_mock_conn()

    with patch("db._get_conn", return_value=conn):
        upsert_chunk(
            source_type="codebase",
            source_id="src/foo.py",
            chunk_index=0,
            content="def foo(): pass",
            embedding=[0.1] * 768,
            metadata={"lang": "python"},
        )

    assert cursor.execute.called
    assert conn.commit.called
```

- [ ] **Step 2: Run test to confirm it fails**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
pytest tests/test_rag_db.py -v 2>&1 | head -30
```

Expected: `ModuleNotFoundError: No module named 'db'`

- [ ] **Step 3: Write db.py**

Create `rag-mcp/db.py`:

```python
"""pgvector connection and search/upsert helpers."""
from __future__ import annotations

import json
import os
from typing import Any

import psycopg2
from pgvector.psycopg2 import register_vector

from models import SearchResult

_DATABASE_URL = os.environ.get("DATABASE_URL", "")


def _get_conn():
    """Return a new psycopg2 connection with pgvector registered."""
    conn = psycopg2.connect(_DATABASE_URL)
    register_vector(conn)
    return conn


def apply_migration(sql_path: str) -> None:
    """Run a SQL file against the database (idempotent — uses IF NOT EXISTS)."""
    with open(sql_path) as f:
        sql = f.read()
    conn = _get_conn()
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
        conn.commit()
    finally:
        conn.close()


def search_chunks(
    source_type: str,
    query_embedding: list[float],
    top_k: int = 5,
) -> list[SearchResult]:
    """Return top-k chunks by cosine similarity filtered by source_type.

    Args:
        source_type: 'codebase' | 'memory' | 'docs'
        query_embedding: Embedding vector for the query string.
        top_k: Maximum number of results to return.

    Returns:
        List of SearchResult ordered by score descending.
    """
    sql = """
        SELECT content, source_id, chunk_index,
               1 - (embedding <=> %s::vector) AS score,
               metadata
        FROM rag_embeddings
        WHERE source_type = %s
        ORDER BY embedding <=> %s::vector
        LIMIT %s
    """
    conn = _get_conn()
    try:
        with conn.cursor() as cur:
            cur.execute(sql, (query_embedding, source_type, query_embedding, top_k))
            rows = cur.fetchall()
    finally:
        conn.close()

    return [
        SearchResult(
            content=row[0],
            source_id=row[1],
            chunk_index=row[2],
            score=float(row[3]),
            metadata=row[4] if isinstance(row[4], dict) else json.loads(row[4] or "{}"),
        )
        for row in rows
    ]


def upsert_chunk(
    source_type: str,
    source_id: str,
    chunk_index: int,
    content: str,
    embedding: list[float],
    metadata: dict[str, Any] | None = None,
) -> None:
    """Insert or update a single chunk row (idempotent by source_type+source_id+chunk_index)."""
    sql = """
        INSERT INTO rag_embeddings (source_type, source_id, chunk_index, content, embedding, metadata)
        VALUES (%s, %s, %s, %s, %s::vector, %s)
        ON CONFLICT (source_type, source_id, chunk_index)
        DO UPDATE SET
            content   = EXCLUDED.content,
            embedding = EXCLUDED.embedding,
            metadata  = EXCLUDED.metadata
    """
    conn = _get_conn()
    try:
        with conn.cursor() as cur:
            cur.execute(
                sql,
                (
                    source_type,
                    source_id,
                    chunk_index,
                    content,
                    embedding,
                    json.dumps(metadata or {}),
                ),
            )
        conn.commit()
    finally:
        conn.close()


def delete_stale_chunks(source_type: str, live_source_ids: list[str]) -> int:
    """Delete chunks whose source_id is no longer in the live set.

    Used by indexer --clean flag to remove embeddings for deleted files.

    Returns:
        Number of rows deleted.
    """
    if not live_source_ids:
        return 0
    sql = """
        DELETE FROM rag_embeddings
        WHERE source_type = %s
          AND source_id NOT IN %s
    """
    conn = _get_conn()
    try:
        with conn.cursor() as cur:
            cur.execute(sql, (source_type, tuple(live_source_ids)))
            deleted = cur.rowcount
        conn.commit()
    finally:
        conn.close()
    return deleted
```

- [ ] **Step 4: Update test to match actual search_chunks signature**

The test calls `search_chunks("codebase", "database", top_k=3)` (passing string query) but `db.py` expects an embedding vector. Update the test to pass a vector:

Edit `tests/test_rag_db.py` — change the test call:

```python
def test_search_chunks_returns_search_results():
    """search_chunks() maps pgvector rows to SearchResult objects."""
    from db import search_chunks
    from models import SearchResult

    row = ("hello world", "src/main.py", 0, 0.92, json.dumps({"lang": "python"}))
    conn, cursor = make_mock_conn(rows=[row])

    embedding = [0.0] * 768

    with patch("db._get_conn", return_value=conn):
        results = search_chunks("codebase", embedding, top_k=3)

    assert len(results) == 1
    r = results[0]
    assert isinstance(r, SearchResult)
    assert r.content == "hello world"
    assert r.source_id == "src/main.py"
    assert r.chunk_index == 0
    assert abs(r.score - 0.92) < 0.001
    assert r.metadata == {"lang": "python"}


def test_search_chunks_filters_by_source_type():
    """search_chunks() passes source_type as a WHERE filter."""
    from db import search_chunks

    conn, cursor = make_mock_conn(rows=[])
    embedding = [0.0] * 768

    with patch("db._get_conn", return_value=conn):
        search_chunks("memory", embedding, top_k=5)

    sql_called = cursor.execute.call_args[0][0]
    assert "source_type" in sql_called
```

- [ ] **Step 5: Run tests from rag-mcp directory**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
pip install psycopg2-binary pgvector --quiet
PYTHONPATH=rag-mcp pytest tests/test_rag_db.py -v
```

Expected: 3 tests PASS.

- [ ] **Step 6: Commit**

```bash
git add rag-mcp/db.py tests/test_rag_db.py
git commit -m "feat(rag): add db.py with pgvector search and upsert helpers"
```

---

### Task 3: embedder.py — Ollama/vLLM/OpenAI embedding backends

**Files:**
- Create: `rag-mcp/embedder.py`
- Create: `tests/test_rag_embedder.py`

- [ ] **Step 1: Write the failing tests**

Create `tests/test_rag_embedder.py`:

```python
"""Tests for rag-mcp/embedder.py — all backends mocked."""
import sys, os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "rag-mcp"))

import json
from unittest.mock import MagicMock, patch
import pytest


# ── Ollama ────────────────────────────────────────────────────────────────────

def test_ollama_embed_sends_correct_request(monkeypatch):
    """Ollama backend POSTs to /api/embeddings with model+prompt."""
    monkeypatch.setenv("EMBED_BACKEND", "ollama")
    monkeypatch.setenv("OLLAMA_BASE_URL", "http://192.168.1.10:11434")
    monkeypatch.setenv("OLLAMA_MODEL", "nomic-embed-text")

    fake_resp = MagicMock()
    fake_resp.json.return_value = {"embedding": [0.1, 0.2, 0.3]}
    fake_resp.raise_for_status = MagicMock()

    with patch("requests.post", return_value=fake_resp) as mock_post:
        from embedder import Embedder
        e = Embedder()
        result = e.embed("hello")

    mock_post.assert_called_once()
    call_kwargs = mock_post.call_args
    url = call_kwargs[0][0] if call_kwargs[0] else call_kwargs[1]["url"]
    assert "/api/embeddings" in url
    body = call_kwargs[1].get("json") or call_kwargs[0][1]
    assert body["model"] == "nomic-embed-text"
    assert body["prompt"] == "hello"
    assert result == [0.1, 0.2, 0.3]


def test_ollama_embed_raises_embedder_error_on_network_failure(monkeypatch):
    """Ollama backend wraps requests.exceptions.ConnectionError as EmbedderError."""
    monkeypatch.setenv("EMBED_BACKEND", "ollama")
    monkeypatch.setenv("OLLAMA_BASE_URL", "http://192.168.1.10:11434")
    monkeypatch.setenv("OLLAMA_MODEL", "nomic-embed-text")

    import requests as req_mod
    with patch("requests.post", side_effect=req_mod.exceptions.ConnectionError("refused")):
        from embedder import Embedder, EmbedderError
        e = Embedder()
        with pytest.raises(EmbedderError, match="refused"):
            e.embed("hello")


# ── vLLM ─────────────────────────────────────────────────────────────────────

def test_vllm_embed_sends_correct_request(monkeypatch):
    """vLLM backend POSTs to /v1/embeddings with model+input."""
    monkeypatch.setenv("EMBED_BACKEND", "vllm")
    monkeypatch.setenv("VLLM_BASE_URL", "http://192.168.1.10:8000")
    monkeypatch.setenv("VLLM_MODEL", "BAAI/bge-m3")

    fake_resp = MagicMock()
    fake_resp.json.return_value = {"data": [{"embedding": [0.5, 0.6]}]}
    fake_resp.raise_for_status = MagicMock()

    with patch("requests.post", return_value=fake_resp) as mock_post:
        import importlib
        import embedder as emb_mod
        importlib.reload(emb_mod)
        e = emb_mod.Embedder()
        result = e.embed("world")

    mock_post.assert_called_once()
    call_kwargs = mock_post.call_args
    url = call_kwargs[0][0] if call_kwargs[0] else call_kwargs[1]["url"]
    assert "/v1/embeddings" in url
    body = call_kwargs[1].get("json") or call_kwargs[0][1]
    assert body["model"] == "BAAI/bge-m3"
    assert body["input"] == "world"
    assert result == [0.5, 0.6]


# ── embed_batch ───────────────────────────────────────────────────────────────

def test_embed_batch_returns_list_of_embeddings(monkeypatch):
    """embed_batch() calls embed() for each text and returns a list."""
    monkeypatch.setenv("EMBED_BACKEND", "ollama")
    monkeypatch.setenv("OLLAMA_BASE_URL", "http://localhost:11434")
    monkeypatch.setenv("OLLAMA_MODEL", "nomic-embed-text")

    fake_resp = MagicMock()
    fake_resp.raise_for_status = MagicMock()

    call_count = 0

    def fake_post(url, json=None, timeout=None):
        nonlocal call_count
        call_count += 1
        fake_resp.json.return_value = {"embedding": [float(call_count)] * 3}
        return fake_resp

    with patch("requests.post", side_effect=fake_post):
        import importlib
        import embedder as emb_mod
        importlib.reload(emb_mod)
        e = emb_mod.Embedder()
        results = e.embed_batch(["a", "b", "c"])

    assert len(results) == 3
    assert results[0] == [1.0, 1.0, 1.0]
    assert results[1] == [2.0, 2.0, 2.0]
    assert results[2] == [3.0, 3.0, 3.0]
```

- [ ] **Step 2: Run tests to confirm they fail**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_embedder.py -v 2>&1 | head -20
```

Expected: `ModuleNotFoundError: No module named 'embedder'`

- [ ] **Step 3: Write embedder.py**

Create `rag-mcp/embedder.py`:

```python
"""Embedding client with Ollama, vLLM, and OpenAI backends.

Backend is selected by the EMBED_BACKEND environment variable:
  - 'ollama'  (default): POST /api/embeddings to OLLAMA_BASE_URL
  - 'vllm':              POST /v1/embeddings to VLLM_BASE_URL
  - 'openai':            OpenAI SDK with OPENAI_API_KEY
"""
from __future__ import annotations

import os

import requests
import requests.exceptions

from models import EmbedderError

_BACKEND = os.environ.get("EMBED_BACKEND", "ollama")
_OLLAMA_BASE = os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434").rstrip("/")
_OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "nomic-embed-text")
_VLLM_BASE = os.environ.get("VLLM_BASE_URL", "http://localhost:8000").rstrip("/")
_VLLM_MODEL = os.environ.get("VLLM_MODEL", "BAAI/bge-m3")
_OPENAI_MODEL = os.environ.get("OPENAI_EMBED_MODEL", "text-embedding-3-small")
_TIMEOUT = 30


class Embedder:
    """Generates text embeddings using the configured backend."""

    def embed(self, text: str) -> list[float]:
        """Return the embedding vector for a single text string.

        Raises:
            EmbedderError: If the backend is unreachable or returns an error.
        """
        try:
            if _BACKEND == "ollama":
                return self._embed_ollama(text)
            elif _BACKEND == "vllm":
                return self._embed_vllm(text)
            elif _BACKEND == "openai":
                return self._embed_openai(text)
            else:
                raise EmbedderError(f"Unknown EMBED_BACKEND: {_BACKEND!r}")
        except EmbedderError:
            raise
        except Exception as exc:
            raise EmbedderError(f"Embedding failed ({_BACKEND}): {exc}") from exc

    def embed_batch(self, texts: list[str]) -> list[list[float]]:
        """Return embeddings for a list of texts.

        Calls embed() per item. Raises EmbedderError on first failure.
        """
        return [self.embed(t) for t in texts]

    # ── backends ──────────────────────────────────────────────────────────────

    def _embed_ollama(self, text: str) -> list[float]:
        url = f"{_OLLAMA_BASE}/api/embeddings"
        try:
            resp = requests.post(
                url,
                json={"model": _OLLAMA_MODEL, "prompt": text},
                timeout=_TIMEOUT,
            )
            resp.raise_for_status()
        except requests.exceptions.RequestException as exc:
            raise EmbedderError(str(exc)) from exc
        return resp.json()["embedding"]

    def _embed_vllm(self, text: str) -> list[float]:
        url = f"{_VLLM_BASE}/v1/embeddings"
        try:
            resp = requests.post(
                url,
                json={"model": _VLLM_MODEL, "input": text},
                timeout=_TIMEOUT,
            )
            resp.raise_for_status()
        except requests.exceptions.RequestException as exc:
            raise EmbedderError(str(exc)) from exc
        return resp.json()["data"][0]["embedding"]

    def _embed_openai(self, text: str) -> list[float]:
        try:
            from openai import OpenAI
        except ImportError as exc:
            raise EmbedderError("openai package not installed") from exc
        client = OpenAI()
        resp = client.embeddings.create(model=_OPENAI_MODEL, input=text)
        return resp.data[0].embedding
```

- [ ] **Step 4: Run tests**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_embedder.py -v
```

Expected: 5 tests PASS.

- [ ] **Step 5: Commit**

```bash
git add rag-mcp/embedder.py tests/test_rag_embedder.py
git commit -m "feat(rag): add embedder.py with Ollama/vLLM/OpenAI backends"
```

---

### Task 4: indexer.py — CLI for chunking and upserting sources

**Files:**
- Create: `rag-mcp/indexer.py`
- Create: `tests/test_rag_indexer.py`

- [ ] **Step 1: Write the failing tests**

Create `tests/test_rag_indexer.py`:

```python
"""Tests for rag-mcp/indexer.py — chunking logic and upsert behaviour."""
import sys, os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "rag-mcp"))

import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch, call
import pytest


# ── Chunking ──────────────────────────────────────────────────────────────────

def test_chunk_code_produces_correct_size_and_overlap():
    """Code chunks are 40 lines with 10-line overlap."""
    from indexer import chunk_code

    lines = [f"line {i}" for i in range(100)]
    text = "\n".join(lines)
    chunks = chunk_code(text, chunk_size=40, overlap=10)

    # First chunk: lines 0-39
    assert chunks[0].startswith("line 0")
    # Second chunk starts at line 30 (40-10 overlap)
    assert chunks[1].startswith("line 30")
    # All chunks have at most 40 lines
    for c in chunks:
        assert len(c.splitlines()) <= 40


def test_chunk_text_produces_correct_token_count():
    """Text chunks are ~500 tokens with 50-token overlap (approximated by words)."""
    from indexer import chunk_text

    # ~1200 words → should produce multiple chunks
    words = ["word"] * 1200
    text = " ".join(words)
    chunks = chunk_text(text, chunk_size=500, overlap=50)

    assert len(chunks) >= 2
    # Each chunk is at most chunk_size+overlap words
    for c in chunks:
        assert len(c.split()) <= 560  # 500 + small tolerance


def test_chunk_code_single_chunk_when_file_is_small():
    """Files shorter than chunk_size produce exactly one chunk."""
    from indexer import chunk_code

    text = "\n".join(f"line {i}" for i in range(10))
    chunks = chunk_code(text)
    assert len(chunks) == 1


# ── Indexing ──────────────────────────────────────────────────────────────────

def test_index_codebase_upserts_chunks_for_each_file():
    """index_codebase() calls upsert_chunk for each chunk of each file."""
    from indexer import index_codebase

    with tempfile.TemporaryDirectory() as tmpdir:
        (Path(tmpdir) / "a.py").write_text("\n".join(f"line {i}" for i in range(5)))
        (Path(tmpdir) / "b.py").write_text("\n".join(f"line {i}" for i in range(5)))

        mock_embedder = MagicMock()
        mock_embedder.embed.return_value = [0.1] * 768

        with patch("indexer.upsert_chunk") as mock_upsert:
            index_codebase(tmpdir, mock_embedder, extensions=["py"])

    # 2 files × 1 chunk each = 2 upsert calls
    assert mock_upsert.call_count == 2
    calls_args = [c[1] for c in mock_upsert.call_args_list]
    source_ids = {a["source_id"] for a in calls_args}
    assert any("a.py" in sid for sid in source_ids)
    assert any("b.py" in sid for sid in source_ids)


def test_index_codebase_is_idempotent():
    """Calling index_codebase twice on the same file calls upsert (not insert) — upsert handles idempotency."""
    from indexer import index_codebase

    with tempfile.TemporaryDirectory() as tmpdir:
        (Path(tmpdir) / "c.py").write_text("x = 1\n")

        mock_embedder = MagicMock()
        mock_embedder.embed.return_value = [0.0] * 768

        with patch("indexer.upsert_chunk") as mock_upsert:
            index_codebase(tmpdir, mock_embedder, extensions=["py"])
            index_codebase(tmpdir, mock_embedder, extensions=["py"])

    # Both runs call upsert — idempotency is enforced by ON CONFLICT in db.py
    assert mock_upsert.call_count == 2
```

- [ ] **Step 2: Run tests to confirm they fail**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_indexer.py -v 2>&1 | head -20
```

Expected: `ModuleNotFoundError: No module named 'indexer'`

- [ ] **Step 3: Write indexer.py**

Create `rag-mcp/indexer.py`:

```python
"""Indexer CLI — chunks source files and upserts embeddings into pgvector.

Usage:
    python indexer.py --source codebase --path /path/to/repo [--ext py,ts,go] [--clean]
    python indexer.py --source docs --path /path/to/docs [--ext md,txt,rst]
    python indexer.py --source memory --db /path/to/memory.db

Environment variables required:
    DATABASE_URL  — Postgres connection string
    EMBED_BACKEND, OLLAMA_BASE_URL, etc. — see embedder.py
"""
from __future__ import annotations

import argparse
import logging
import os
import sys
from pathlib import Path

from db import upsert_chunk, delete_stale_chunks
from embedder import Embedder, EmbedderError
from models import EmbedderError  # re-exported for tests

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
log = logging.getLogger(__name__)

_CHUNK_SIZE_CODE = int(os.environ.get("CHUNK_SIZE_CODE", "40"))
_CHUNK_OVERLAP_CODE = int(os.environ.get("CHUNK_OVERLAP_CODE", "10"))
_CHUNK_SIZE_TEXT = int(os.environ.get("CHUNK_SIZE_TEXT", "500"))
_CHUNK_OVERLAP_TEXT = int(os.environ.get("CHUNK_OVERLAP_TEXT", "50"))


def chunk_code(text: str, chunk_size: int = _CHUNK_SIZE_CODE, overlap: int = _CHUNK_OVERLAP_CODE) -> list[str]:
    """Split text into chunks of chunk_size lines with overlap lines of overlap."""
    lines = text.splitlines()
    if len(lines) <= chunk_size:
        return [text]
    chunks = []
    step = chunk_size - overlap
    start = 0
    while start < len(lines):
        end = min(start + chunk_size, len(lines))
        chunks.append("\n".join(lines[start:end]))
        if end == len(lines):
            break
        start += step
    return chunks


def chunk_text(text: str, chunk_size: int = _CHUNK_SIZE_TEXT, overlap: int = _CHUNK_OVERLAP_TEXT) -> list[str]:
    """Split text into chunks of ~chunk_size words with ~overlap words of overlap.

    Uses words as a proxy for tokens (1 word ≈ 1.3 tokens on average).
    """
    words = text.split()
    if len(words) <= chunk_size:
        return [text]
    chunks = []
    step = chunk_size - overlap
    start = 0
    while start < len(words):
        end = min(start + chunk_size, len(words))
        chunks.append(" ".join(words[start:end]))
        if end == len(words):
            break
        start += step
    return chunks


def index_codebase(
    path: str,
    embedder: Embedder,
    extensions: list[str] | None = None,
    clean: bool = False,
) -> None:
    """Index all source files under path into the 'codebase' source type."""
    exts = {f".{e.lstrip('.')}" for e in (extensions or ["py", "ts", "js", "go", "java"])}
    root = Path(path)
    live_ids: list[str] = []

    for fpath in sorted(root.rglob("*")):
        if fpath.suffix not in exts or not fpath.is_file():
            continue
        source_id = str(fpath)
        live_ids.append(source_id)
        text = fpath.read_text(errors="replace")
        chunks = chunk_code(text)
        for i, chunk in enumerate(chunks):
            try:
                embedding = embedder.embed(chunk)
            except EmbedderError as exc:
                log.warning("Skipping %s chunk %d: %s", source_id, i, exc)
                continue
            upsert_chunk(
                source_type="codebase",
                source_id=source_id,
                chunk_index=i,
                content=chunk,
                embedding=embedding,
                metadata={"ext": fpath.suffix, "path": source_id},
            )
            log.info("Indexed codebase %s chunk %d", source_id, i)

    if clean:
        deleted = delete_stale_chunks("codebase", live_ids)
        log.info("Cleaned %d stale codebase chunks", deleted)


def index_docs(
    path: str,
    embedder: Embedder,
    extensions: list[str] | None = None,
    clean: bool = False,
) -> None:
    """Index documentation files under path into the 'docs' source type."""
    exts = {f".{e.lstrip('.')}" for e in (extensions or ["md", "txt", "rst"])}
    root = Path(path)
    live_ids: list[str] = []

    for fpath in sorted(root.rglob("*")):
        if fpath.suffix not in exts or not fpath.is_file():
            continue
        source_id = str(fpath)
        live_ids.append(source_id)
        text = fpath.read_text(errors="replace")
        chunks = chunk_text(text)
        for i, chunk in enumerate(chunks):
            try:
                embedding = embedder.embed(chunk)
            except EmbedderError as exc:
                log.warning("Skipping %s chunk %d: %s", source_id, i, exc)
                continue
            upsert_chunk(
                source_type="docs",
                source_id=source_id,
                chunk_index=i,
                content=chunk,
                embedding=embedding,
                metadata={"ext": fpath.suffix, "path": source_id},
            )
            log.info("Indexed docs %s chunk %d", source_id, i)

    if clean:
        deleted = delete_stale_chunks("docs", live_ids)
        log.info("Cleaned %d stale docs chunks", deleted)


def index_memory(db_path: str, embedder: Embedder) -> None:
    """Index past pipeline runs from the MemoryStore SQLite database.

    Reads the 'runs' table and indexes PRD + design + summary for each run.
    """
    import sqlite3

    conn = sqlite3.connect(db_path)
    try:
        cur = conn.execute(
            "SELECT id, prd, design, summary, created_at FROM runs WHERE summary IS NOT NULL"
        )
        for row in cur.fetchall():
            run_id, prd, design, summary, created_at = row
            parts = [
                ("prd", prd or ""),
                ("design", design or ""),
                ("summary", summary or ""),
            ]
            chunk_index = 0
            for part_name, text in parts:
                if not text.strip():
                    continue
                for chunk in chunk_text(text):
                    try:
                        embedding = embedder.embed(chunk)
                    except EmbedderError as exc:
                        log.warning("Skipping memory run=%s part=%s: %s", run_id, part_name, exc)
                        continue
                    upsert_chunk(
                        source_type="memory",
                        source_id=str(run_id),
                        chunk_index=chunk_index,
                        content=chunk,
                        embedding=embedding,
                        metadata={"part": part_name, "ts": created_at},
                    )
                    chunk_index += 1
                    log.info("Indexed memory run=%s chunk %d", run_id, chunk_index)
    finally:
        conn.close()


def main() -> None:
    parser = argparse.ArgumentParser(description="RAG indexer — populate pgvector knowledge base")
    parser.add_argument("--source", choices=["codebase", "docs", "memory"], required=True)
    parser.add_argument("--path", help="Root directory for codebase/docs sources")
    parser.add_argument("--db", help="Path to MemoryStore SQLite file (--source memory)")
    parser.add_argument("--ext", help="Comma-separated file extensions to index (e.g. py,ts,go)")
    parser.add_argument("--clean", action="store_true", help="Delete embeddings for removed files")
    args = parser.parse_args()

    embedder = Embedder()
    exts = args.ext.split(",") if args.ext else None

    if args.source == "codebase":
        if not args.path:
            parser.error("--path required for --source codebase")
        index_codebase(args.path, embedder, extensions=exts, clean=args.clean)
    elif args.source == "docs":
        if not args.path:
            parser.error("--path required for --source docs")
        index_docs(args.path, embedder, extensions=exts, clean=args.clean)
    elif args.source == "memory":
        if not args.db:
            parser.error("--db required for --source memory")
        index_memory(args.db, embedder)

    log.info("Indexing complete.")


if __name__ == "__main__":
    main()
```

- [ ] **Step 4: Run tests**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_indexer.py -v
```

Expected: 6 tests PASS.

- [ ] **Step 5: Commit**

```bash
git add rag-mcp/indexer.py tests/test_rag_indexer.py
git commit -m "feat(rag): add indexer CLI with code/text chunking and upsert"
```

---

### Task 5: main.py — FastMCP server with 3 tools and /health

**Files:**
- Create: `rag-mcp/main.py`

- [ ] **Step 1: Write the failing test**

Create `tests/test_rag_main.py`:

```python
"""Tests for rag-mcp/main.py — tool schemas and health endpoint."""
import sys, os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "rag-mcp"))

import json
from unittest.mock import AsyncMock, patch, MagicMock
import pytest


def test_health_endpoint_returns_ok():
    """GET /health returns {"status": "ok"}."""
    from starlette.testclient import TestClient
    os.environ.setdefault("DATABASE_URL", "postgresql://user:pass@localhost:5432/test")
    os.environ.setdefault("EMBED_BACKEND", "ollama")

    with patch("db.search_chunks", return_value=[]):
        import main as main_mod
        app = main_mod.mcp.streamable_http_app()

    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json()["status"] == "ok"


def test_search_codebase_tool_registered():
    """search_codebase tool is registered in the FastMCP server."""
    import main as main_mod

    tool_names = [t.name for t in main_mod.mcp._tool_manager.list_tools()]
    assert "search_codebase" in tool_names
    assert "search_memory" in tool_names
    assert "search_docs" in tool_names
```

- [ ] **Step 2: Run test to confirm it fails**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_main.py -v 2>&1 | head -20
```

Expected: `ModuleNotFoundError: No module named 'main'`

- [ ] **Step 3: Write main.py**

Create `rag-mcp/main.py`:

```python
"""RAG MCP Server — FastMCP server exposing search tools over Streamable HTTP.

Tools:
    search_codebase(query, top_k) — search indexed source code
    search_memory(query, top_k)   — search past pipeline runs
    search_docs(query, top_k)     — search indexed documentation

Transport: Streamable HTTP at /mcp (default FastMCP path)
Health:    GET /health → {"status": "ok"}

Environment variables:
    DATABASE_URL    — Postgres connection string (required)
    EMBED_BACKEND   — ollama | vllm | openai (default: ollama)
    RAG_TOP_K       — default number of results (default: 5)
    (see embedder.py for backend-specific vars)
"""
from __future__ import annotations

import asyncio
import json
import os

from mcp.server.fastmcp import FastMCP
from starlette.requests import Request
from starlette.responses import JSONResponse

from db import search_chunks
from embedder import Embedder, EmbedderError

_TOP_K = int(os.environ.get("RAG_TOP_K", "5"))
_embedder = Embedder()

mcp = FastMCP("rag-mcp-server")


@mcp.custom_route("/health", methods=["GET"])
async def health(request: Request) -> JSONResponse:
    """Liveness probe — used by docker-compose healthcheck."""
    return JSONResponse({"status": "ok"})


@mcp.tool()
async def search_codebase(query: str, top_k: int = _TOP_K) -> str:
    """Search the indexed codebase for relevant code chunks.

    Args:
        query: Natural-language description of what you're looking for.
        top_k: Maximum number of results to return (default 5).

    Returns:
        JSON array of SearchResult objects with content, source_id, score, and metadata.
    """
    try:
        embedding = await asyncio.to_thread(_embedder.embed, query)
        results = await asyncio.to_thread(search_chunks, "codebase", embedding, top_k)
        return json.dumps([r.model_dump() for r in results])
    except EmbedderError as exc:
        return json.dumps({"error": f"embedder unavailable: {exc}", "results": []})
    except Exception as exc:
        return json.dumps({"error": str(exc), "results": []})


@mcp.tool()
async def search_memory(query: str, top_k: int = _TOP_K) -> str:
    """Search past pipeline runs for relevant PRD, design, or summary excerpts.

    Args:
        query: Natural-language description of what you're looking for.
        top_k: Maximum number of results to return (default 5).

    Returns:
        JSON array of SearchResult objects with content, source_id (run_id), score, and metadata.
    """
    try:
        embedding = await asyncio.to_thread(_embedder.embed, query)
        results = await asyncio.to_thread(search_chunks, "memory", embedding, top_k)
        return json.dumps([r.model_dump() for r in results])
    except EmbedderError as exc:
        return json.dumps({"error": f"embedder unavailable: {exc}", "results": []})
    except Exception as exc:
        return json.dumps({"error": str(exc), "results": []})


@mcp.tool()
async def search_docs(query: str, top_k: int = _TOP_K) -> str:
    """Search indexed documentation for relevant content.

    Args:
        query: Natural-language description of what you're looking for.
        top_k: Maximum number of results to return (default 5).

    Returns:
        JSON array of SearchResult objects with content, source_id (file path), score, and metadata.
    """
    try:
        embedding = await asyncio.to_thread(_embedder.embed, query)
        results = await asyncio.to_thread(search_chunks, "docs", embedding, top_k)
        return json.dumps([r.model_dump() for r in results])
    except EmbedderError as exc:
        return json.dumps({"error": f"embedder unavailable: {exc}", "results": []})
    except Exception as exc:
        return json.dumps({"error": str(exc), "results": []})


if __name__ == "__main__":
    mcp.run(transport="streamable-http")
```

- [ ] **Step 4: Run tests**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
PYTHONPATH=rag-mcp pytest tests/test_rag_main.py -v
```

Expected: 2 tests PASS.

- [ ] **Step 5: Commit**

```bash
git add rag-mcp/main.py tests/test_rag_main.py
git commit -m "feat(rag): add FastMCP server with search_codebase/memory/docs tools"
```

---

### Task 6: Dockerfile + requirements.txt + docker-compose.yml

**Files:**
- Create: `rag-mcp/requirements.txt`
- Create: `rag-mcp/Dockerfile`
- Create: `rag-mcp/docker-compose.yml`

- [ ] **Step 1: Write requirements.txt**

Create `rag-mcp/requirements.txt`:

```
mcp>=1.0.0
uvicorn>=0.30.0
psycopg2-binary>=2.9.9
pgvector>=0.2.5
requests>=2.31.0
openai>=1.30.0
```

- [ ] **Step 2: Write Dockerfile**

Create `rag-mcp/Dockerfile`:

```dockerfile
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8001

CMD ["python", "-m", "uvicorn", "main:mcp_http_app", "--host", "0.0.0.0", "--port", "8001"]
```

Wait — `mcp.streamable_http_app()` returns the Starlette app. We need to expose it as a uvicorn-runnable app. Update `main.py` to expose the app object:

Edit `rag-mcp/main.py` — add these lines at the bottom (before `if __name__ == "__main__":`):

```python
# ASGI app for uvicorn — used by Dockerfile CMD
mcp_http_app = mcp.streamable_http_app()
```

- [ ] **Step 3: Verify the app object is importable**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
DATABASE_URL=postgresql://x:x@localhost/x PYTHONPATH=rag-mcp python -c "
from main import mcp_http_app
print(type(mcp_http_app).__name__)
"
```

Expected: `Starlette`

- [ ] **Step 4: Write docker-compose.yml**

Create `rag-mcp/docker-compose.yml`:

```yaml
version: "3.8"

services:
  rag-mcp:
    build:
      context: .
    ports:
      - "8001:8001"
    environment:
      DATABASE_URL: ${DATABASE_URL:-postgresql://user:pass@db:5432/dbname}
      EMBED_BACKEND: ${EMBED_BACKEND:-ollama}
      OLLAMA_BASE_URL: ${OLLAMA_BASE_URL:-http://192.168.1.x:11434}
      OLLAMA_MODEL: ${OLLAMA_MODEL:-nomic-embed-text}
      # VLLM_BASE_URL: http://192.168.1.x:8000
      # VLLM_MODEL: BAAI/bge-m3
      RAG_TOP_K: ${RAG_TOP_K:-5}
    healthcheck:
      test: ["CMD", "python", "-c",
             "import urllib.request; urllib.request.urlopen('http://localhost:8001/health')"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 15s
    restart: unless-stopped
```

- [ ] **Step 5: Build the Docker image**

```bash
cd /home/wanleung/Projects/ai-software-house/rag-mcp
docker build -t rag-mcp:latest . 2>&1 | tail -5
```

Expected: `Successfully built` or `=> exporting to image` (buildx).

- [ ] **Step 6: Update config.yaml with RAG entry example**

Edit `config.yaml` — add a commented-out RAG server entry in the MCP servers section. Find the `mcp:` block and add:

```yaml
mcp:
  servers:
    # RAG MCP server — enables search_codebase, search_memory, search_docs tools.
    # Start with: cd rag-mcp && docker compose up -d
    # Then apply migration: docker exec <container> psql $DATABASE_URL -f migrations/001_create_rag_embeddings.sql
    # - name: rag
    #   type: http
    #   url: "http://localhost:8001/mcp"
```

- [ ] **Step 7: Commit**

```bash
git add rag-mcp/requirements.txt rag-mcp/Dockerfile rag-mcp/docker-compose.yml
git add rag-mcp/main.py config.yaml
git commit -m "feat(rag): add Dockerfile, requirements, docker-compose, and config example"
```

---

### Task 7: Agent integration — EngineerAgent, ArchitectAgent, QAEngineerAgent + orchestrator

**Files:**
- Modify: `agents/engineer.py`
- Modify: `agents/architect.py`
- Modify: `agents/qa_engineer.py`
- Modify: `orchestrator.py`

- [ ] **Step 1: Write the failing test**

Add to `tests/test_pipeline.py` (or create `tests/test_rag_agent_integration.py`):

```python
"""Tests for RAG tool_registry wiring in agents and orchestrator."""
import sys, os
from unittest.mock import MagicMock, patch
import pytest

from agents.engineer import EngineerAgent
from agents.architect import ArchitectAgent
from agents.qa_engineer import QAEngineerAgent


def make_mock_registry():
    reg = MagicMock()
    reg.list_tools.return_value = []
    return reg


def test_engineer_uses_call_with_tools_when_registry_provided():
    """EngineerAgent uses call_with_tools when tool_registry is set."""
    agent = EngineerAgent(model="gpt-4o-mini", tool_registry=make_mock_registry())
    assert agent._tool_registry is not None

    with patch.object(agent, "call_with_tools", return_value="### FILE: foo.py\n```\npass\n```") as mock_cwt, \
         patch.object(agent, "call"):
        agent.run_module("design", {"name": "foo", "description": "bar"})
        assert mock_cwt.called


def test_engineer_uses_call_when_no_registry():
    """EngineerAgent uses call() when tool_registry is None."""
    agent = EngineerAgent(model="gpt-4o-mini")
    assert agent._tool_registry is None

    with patch.object(agent, "call", return_value="### FILE: foo.py\n```\npass\n```") as mock_call, \
         patch.object(agent, "call_with_tools"):
        agent.run_module("design", {"name": "foo", "description": "bar"})
        assert mock_call.called


def test_architect_uses_call_with_tools_when_registry_provided():
    """ArchitectAgent uses call_with_tools when tool_registry is set."""
    agent = ArchitectAgent(model="gpt-4o-mini", tool_registry=make_mock_registry())
    with patch.object(agent, "call_with_tools", return_value="# Design\n## Implementation Modules\n- foo") as mock_cwt, \
         patch.object(agent, "call"):
        agent.run("prd text")
        assert mock_cwt.called


def test_qa_engineer_uses_call_with_tools_when_registry_provided():
    """QAEngineerAgent uses call_with_tools when tool_registry is set."""
    agent = QAEngineerAgent(model="gpt-4o-mini", tool_registry=make_mock_registry())
    with patch.object(agent, "call_with_tools", return_value="## Test Plan\n```python\npass\n```") as mock_cwt, \
         patch.object(agent, "call"):
        agent.run(files={"foo.py": "x=1"}, prd="prd", project_name="proj")
        assert mock_cwt.called
```

- [ ] **Step 2: Run test to confirm it fails**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
pytest tests/test_rag_agent_integration.py -v 2>&1 | head -30
```

Expected: `TypeError: __init__() got an unexpected keyword argument 'tool_registry'`

- [ ] **Step 3: Update EngineerAgent**

Edit `agents/engineer.py` — add imports and `__init__`:

After the existing imports, add:
```python
from tools import ToolRegistry
```

Add `__init__` method to `EngineerAgent` before `run_module`:
```python
def __init__(self, *args, tool_registry: "ToolRegistry | None" = None, **kwargs) -> None:
    super().__init__(*args, **kwargs)
    self._tool_registry = tool_registry
```

In `run_module()`, replace:
```python
response = self.call(prompt)
```
with:
```python
if self._tool_registry is not None:
    rag_hint = (
        "\n\nYou have access to RAG search tools: `search_codebase` and `search_docs`. "
        "Use them to find relevant existing code patterns and documentation before implementing."
    )
    response = self.call_with_tools(prompt + rag_hint, tools=self._tool_registry)
else:
    response = self.call(prompt)
```

- [ ] **Step 4: Update ArchitectAgent**

Edit `agents/architect.py` — add imports and `__init__`:

After existing imports, add:
```python
from tools import ToolRegistry
```

Add `__init__` before `run`:
```python
def __init__(self, *args, tool_registry: "ToolRegistry | None" = None, **kwargs) -> None:
    super().__init__(*args, **kwargs)
    self._tool_registry = tool_registry
```

In `run()`, replace:
```python
design = self.call(prompt)
```
with:
```python
if self._tool_registry is not None:
    rag_hint = (
        "\n\nYou have access to RAG search tools: `search_memory` and `search_docs`. "
        "Use them to find relevant past designs and documentation before producing the system design."
    )
    design = self.call_with_tools(prompt + rag_hint, tools=self._tool_registry)
else:
    design = self.call(prompt)
```

- [ ] **Step 5: Update QAEngineerAgent**

Edit `agents/qa_engineer.py` — add imports and `__init__`:

After existing imports, add:
```python
from tools import ToolRegistry
```

Add `__init__` before `run`:
```python
def __init__(self, *args, tool_registry: "ToolRegistry | None" = None, **kwargs) -> None:
    super().__init__(*args, **kwargs)
    self._tool_registry = tool_registry
```

In `run()`, find the `response = self.call(prompt)` line and replace with:
```python
if self._tool_registry is not None:
    rag_hint = (
        "\n\nYou have access to the `search_codebase` RAG tool. "
        "Use it to find relevant existing code patterns before writing tests."
    )
    response = self.call_with_tools(prompt + rag_hint, tools=self._tool_registry)
else:
    response = self.call(prompt)
```

- [ ] **Step 6: Update orchestrator.py**

In `orchestrator.py`, find the `__init__` method of `Pipeline` (or equivalent). After the block that builds `tool_registry`:

```python
if mcp_servers:
    mcp_registry = MCPToolRegistry(mcp_servers)
    tool_registry = CombinedToolRegistry(builtin_tools, mcp_registry)
else:
    tool_registry = builtin_tools
```

Add the RAG registry extraction:

```python
# Extract RAG server for agents that benefit from retrieval (no builtin tools mixed in)
rag_servers = [s for s in mcp_servers if s.get("name") == "rag"]
rag_registry = MCPToolRegistry(rag_servers) if rag_servers else None
```

Then update the 3 agent constructor calls:

```python
# Before (engineer):
self.engineer = EngineerAgent(model=_model("engineer"), **agent_kwargs)
# After:
self.engineer = EngineerAgent(model=_model("engineer"), tool_registry=rag_registry, **agent_kwargs)

# Before (architect):
self.architect = ArchitectAgent(model=_model("architect"), **agent_kwargs)
# After:
self.architect = ArchitectAgent(model=_model("architect"), tool_registry=rag_registry, **agent_kwargs)

# Before (qa):
self.qa = QAEngineerAgent(model=_model("qa_engineer"), **agent_kwargs)
# After:
self.qa = QAEngineerAgent(model=_model("qa_engineer"), tool_registry=rag_registry, **agent_kwargs)
```

- [ ] **Step 7: Run all tests**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
pytest tests/test_rag_agent_integration.py tests/test_pipeline.py -v 2>&1 | tail -20
```

Expected: All RAG integration tests PASS. Existing pipeline tests still PASS.

- [ ] **Step 8: Run the full test suite to check for regressions**

```bash
cd /home/wanleung/Projects/ai-software-house
source venv/bin/activate
pytest tests/ -v --ignore=tests/test_outlook_provider.py --ignore=tests/test_sse.py -q 2>&1 | tail -20
```

Expected: No new failures compared to pre-task baseline.

- [ ] **Step 9: Commit**

```bash
git add agents/engineer.py agents/architect.py agents/qa_engineer.py orchestrator.py
git add tests/test_rag_agent_integration.py
git commit -m "feat(rag): wire RAG tool_registry into EngineerAgent, ArchitectAgent, QAEngineerAgent and orchestrator"
```

---

### Task 8: Push all commits to origin

- [ ] **Step 1: Verify git log**

```bash
cd /home/wanleung/Projects/ai-software-house
git --no-pager log --oneline origin/master..HEAD
```

Expected: 7–8 commits listed (all RAG tasks from Tasks 1–7).

- [ ] **Step 2: Push**

```bash
git push origin master
```

Expected: `master -> master` with no errors.

- [ ] **Step 3: Confirm remote is up to date**

```bash
git --no-pager log --oneline -3
```

Expected: HEAD matches origin/master.

---

## Notes

- **Embedding dimension**: The migration hard-codes `vector(768)`. If you switch to a model with different dimensions (e.g. `text-embedding-3-small` = 1536), you must drop and recreate the `rag_embeddings` table. Document the model + dimension in your `.env` file.
- **Migration**: Run `psql $DATABASE_URL -f rag-mcp/migrations/001_create_rag_embeddings.sql` before starting the server.
- **Running the indexer**: `cd rag-mcp && DATABASE_URL=... EMBED_BACKEND=ollama OLLAMA_BASE_URL=http://... python indexer.py --source codebase --path /path/to/repo`
- **No RAG = no change**: If `rag` is not in `mcp.servers` config, `rag_registry = None` and all three agents fall back to `call()` — fully backwards compatible.
- **Test files already pre-failing**: `tests/test_outlook_provider.py` and `tests/test_sse.py` were failing before this work — exclude them from regression checks.
