← Back to overview

# SQLite

Serverless database for on-disk text storage.

## What is it?

SQLite is a self-contained, serverless SQL database engine. It requires no separate process, no configuration, no administration. The entire database lives in a single file. It ships with Python's standard library and is the most widely deployed database engine in the world.

## In Membot

The split cartridge format uses SQLite for text storage. The vector index (`_index.npz`) stays in RAM for fast search, while the raw text lives on disk in a SQLite database (`_text.db`). After search identifies the top results by vector similarity, Membot fetches only those texts from SQLite — typically taking ~1 ms per lookup.

This architecture means you can search millions of entries without loading all the text into memory.

## Format comparison

Aspect | Standard (.cart.npz) | Split (_index.npz + _text.db)
--- | --- | ---
Text storage | In-memory (pickle) | On-disk (SQLite)
RAM at 2.4M | ~8 GB | ~1.5 GB (vectors only)
Text lookup | Instant (in RAM) | ~1 ms (disk)
Portability | Single file | Two files
Max practical scale | ~500K entries | Millions+

## Why SQLite?

- Zero setup — no database server to install or manage
- Built-in — ships with Python, no pip install needed
- Portable — the .db file works on any platform
- Concurrent reads — multiple processes can read simultaneously
- Paged access — only loads the data pages actually needed from disk

## Further reading

Brain cartridges — the file format overview
Sign-Zero Encoding — how the index is compressed