yichuan-w/LEANN

🚀 LEANN: A Low-Storage Vector Index

⚡ Real-time embedding computation for large-scale RAG on consumer hardware

Quick Start • Features • Benchmarks • Documentation • Paper

🌟 What is Leann?

Leann revolutionizes Retrieval-Augmented Generation (RAG) by eliminating the storage bottleneck of traditional vector databases. Instead of pre-computing and storing billions of embeddings, Leann dynamically computes embeddings at query time using highly optimized graph-based search algorithms.

🎯 Why Leann?

Traditional RAG systems face a fundamental trade-off:

• 💾 Storage: Storing embeddings for millions of documents requires massive disk space
• 🔄 Freshness: Pre-computed embeddings become stale when documents change
• 💰 Cost: Vector databases are expensive to scale

Leann solves this by:

• ✅ Zero embedding storage - Only graph structure is persisted
• ✅ Real-time computation - Embeddings computed on-demand with ms latency
• ✅ Memory efficient - Runs on consumer hardware (8GB RAM)
• ✅ Always fresh - No stale embeddings, ever

🚀 Quick Start

Installation

1
2
3

git clone https://github.com/yichuan520030910320/Power-RAG.git leann
cd leann
uv sync

30-Second Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

from leann.api import LeannBuilder, LeannSearcher

# 1. Build index (no embeddings stored!)
builder = LeannBuilder(backend_name="diskann")
builder.add_text("Python is a powerful programming language")
builder.add_text("Machine learning transforms industries")  
builder.add_text("Neural networks process complex data")
builder.build_index("knowledge.leann")

# 2. Search with real-time embeddings
searcher = LeannSearcher("knowledge.leann")
results = searcher.search("programming languages", top_k=2)

for result in results:
    print(f"Score: {result['score']:.3f} - {result['text']}")

Run the Demo

1

uv run examples/document_search.py

PDF RAG Demo (using LlamaIndex for document parsing and Leann for indexing/search)

This demo showcases how to build a RAG system for PDF documents using Leann.

1. Place your PDF files (and other supported formats like .docx, .pptx, .xlsx) into the examples/data/ directory.
2. Ensure you have an OPENAI_API_KEY set in your environment variables or in a .env file for the LLM to function.

1

uv run examples/main_cli_example.py

✨ Features

🔥 Core Features

• 📊 Multiple Distance Functions: L2, Cosine, MIPS (Maximum Inner Product Search)
• 🏗️ Pluggable Backends: DiskANN, HNSW/FAISS with unified API
• 🔄 Real-time Embeddings: Dynamic computation using optimized ZMQ servers
• 📈 Scalable Architecture: Handles millions of documents on consumer hardware
• 🎯 Graph Pruning: Advanced techniques for memory-efficient search

🛠️ Technical Highlights

• Zero-copy operations for maximum performance
• SIMD-optimized distance computations (AVX2/AVX512)
• Async embedding pipeline with batched processing
• Memory-mapped indices for fast startup
• Recompute mode for highest accuracy scenarios

🎨 Developer Experience

• Simple Python API - Get started in minutes
• Extensible backend system - Easy to add new algorithms
• Comprehensive examples - From basic usage to production deployment
• Rich debugging tools - Built-in performance profiling

📊 Benchmarks

Memory Usage Comparison

Query Performance

Benchmarks run on AMD Ryzen 7 with 32GB RAM

🏗️ Architecture

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Query Text    │───▶│  Embedding       │───▶│   Graph-based   │
│                 │    │  Computation     │    │     Search      │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                              │                         │
                              ▼                         ▼
                       ┌──────────────┐         ┌──────────────┐
                       │ ZMQ Server   │         │ Pruned Graph │
                       │ (Cached)     │         │ Index        │
                       └──────────────┘         └──────────────┘

Key Components

1. 🧠 Embedding Engine: Real-time transformer inference with caching
2. 📊 Graph Index: Memory-efficient navigation structures
3. 🔄 Search Coordinator: Orchestrates embedding + graph search
4. ⚡ Backend Adapters: Pluggable algorithm implementations

🎓 Supported Models & Backends

🤖 Embedding Models

• sentence-transformers/all-mpnet-base-v2 (default)
• sentence-transformers/all-MiniLM-L6-v2 (lightweight)
• Any HuggingFace sentence-transformer model
• Custom model support via API

🔧 Search Backends

• DiskANN: Microsoft’s billion-scale ANN algorithm
• HNSW: Hierarchical Navigable Small World graphs
• Coming soon: ScaNN, Faiss-IVF, NGT

📏 Distance Functions

• L2: Euclidean distance for precise similarity
• Cosine: Angular similarity for normalized vectors
• MIPS: Maximum Inner Product Search for recommendation systems

🔬 Paper

If you find Leann useful, please cite:

LEANN: A Low-Storage Vector Index

1
2
3
4
5
6
7
8
9

@misc{wang2025leannlowstoragevectorindex,
      title={LEANN: A Low-Storage Vector Index}, 
      author={Yichuan Wang and Shu Liu and Zhifei Li and Yongji Wu and Ziming Mao and Yilong Zhao and Xiao Yan and Zhiying Xu and Yang Zhou and Ion Stoica and Sewon Min and Matei Zaharia and Joseph E. Gonzalez},
      year={2025},
      eprint={2506.08276},
      archivePrefix={arXiv},
      primaryClass={cs.DB},
      url={https://arxiv.org/abs/2506.08276}, 
}

🌍 Use Cases

💼 Enterprise RAG

1
2
3
4
5
6
7

# Handle millions of documents with limited resources
builder = LeannBuilder(
    backend_name="diskann",
    distance_metric="cosine",
    graph_degree=64,
    memory_budget="4GB"
)

🔬 Research & Experimentation

1
2
3
4

# Quick prototyping with different algorithms
for backend in ["diskann", "hnsw"]:
    searcher = LeannSearcher(index_path, backend=backend)
    evaluate_recall(searcher, queries, ground_truth)

🚀 Real-time Applications

1
2
3
4

# Sub-second response times
chat = LeannChat("knowledge.leann")
response = chat.ask("What is quantum computing?")
# Returns in <100ms with recompute mode

🤝 Contributing

We welcome contributions! Leann is built by the community, for the community.

Ways to Contribute

• 🐛 Bug Reports: Found an issue? Let us know!
• 💡 Feature Requests: Have an idea? We’d love to hear it!
• 🔧 Code Contributions: PRs welcome for all skill levels
• 📖 Documentation: Help make Leann more accessible
• 🧪 Benchmarks: Share your performance results

Development Setup

1
2
3
4

git clone https://github.com/yourname/leann
cd leann
uv sync --dev
uv run pytest tests/

Quick Tests

1
2
3
4
5

# Sanity check all distance functions
uv run python tests/sanity_checks/test_distance_functions.py

# Verify L2 implementation
uv run python tests/sanity_checks/test_l2_verification.py

📈 Roadmap

🎯 Q1 2024

• DiskANN backend with MIPS/L2/Cosine support
• HNSW backend integration
• Real-time embedding pipeline
• Memory-efficient graph pruning

🚀 Q2 2024

• Distributed search across multiple nodes
• ScaNN backend support
• Advanced caching strategies
• Kubernetes deployment guides

🌟 Q3 2024

• GPU-accelerated embedding computation
• Approximate distance functions
• Integration with LangChain/LlamaIndex
• Visual similarity search

💬 Community

Join our growing community of researchers and engineers!

• 🐦 Twitter: @LeannAI
• 💬 Discord: Join our server
• 📧 Email: leann@yourcompany.com
• 🐙 GitHub Discussions: Ask questions here

📄 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

• Microsoft Research for the DiskANN algorithm
• Meta AI for FAISS and optimization insights
• HuggingFace for the transformer ecosystem
• Our amazing contributors who make this possible

⭐ Star us on GitHub if Leann is useful for your research or applications!

Made with ❤️ by the Leann team