TL;DR
- TODO: one measured result (for example retrieval hit rate on a set of questions, or answer latency on CPU)
Problem
Documentation is spread across text, Markdown, PDF, image and Excalidraw files. doc-centralizer lets a user explore and understand it through natural-language conversations, locally, with answers grounded in the files.
TODO: who needed it and in what context.
Architecture
flowchart LR
A[data/files/*<br/>txt · md · pdf · image · excalidraw] --> B[Extraction + chunking]
B --> C[Embeddings<br/>sentence-transformers]
C --> D[FAISS index]
E[User question] --> F[Semantic search<br/>top-k chunks]
D --> F
F --> G[LLM prompt<br/>context + history]
G --> H[Phi-3 via llama.cpp<br/>streamed response]
H --> I[Streamlit chat UI]
H --> J[Conversation history<br/>data/history.json]
- Documents are extracted, chunked and embedded on first run; the FAISS index is built once and cached.
- The cache (embeddings, index, chunk metadata) is invalidated and rebuilt whenever a file is added, removed or modified.
- Several conversations can be kept, switched and deleted; their history is persisted and given to the model.
Technical choices & trade-offs
- Fully local generation: quantized Phi-3 mini 128k (
Q4_K_M, about 2.4 GB) through llama.cpp, with an optional NVIDIA GPU for faster inference. - FAISS for semantic search over sentence-transformers embeddings.
- Model weights outside the Docker image:
models/anddata/are mounted as volumes instead of copying multi-gigabyte weights into the image. - Tests without the model: chunking, embedding/FAISS and history logic are tested in CI without the LLM file.
- TODO: the alternatives you considered for each choice and why you rejected them.
Results
TODO: measured results (retrieval quality, answer latency, memory use).
| Metric | Value |
|---|---|
| TODO | TODO |
Limitations & next steps
TODO: known limitations and next steps.