DocMind — RAG-Based Document Chatbot
A retrieval-augmented generation system that answers questions grounded in a custom document set, instead of relying on an LLM's raw memory.
Overview
DocMind is a chatbot that answers questions using a defined set of source documents rather than an LLM's general training data. The goal was simple to state and hard to get right: every answer should be traceable back to a passage in the source material, not a plausible-sounding hallucination.
Problem statement
Generic LLM chat is fast but unreliable for domain-specific questions — it can produce confident, wrong answers when the underlying knowledge isn't in the model's training data. Users working with large document sets needed a way to ask natural-language questions and get answers grounded in their documents, with enough context that the response is verifiable.
Architecture
The system follows a standard retrieval-augmented generation pattern: documents are chunked and embedded, queries are matched against the vector store, and the top-matching chunks are passed to the LLM as grounding context before it generates a response.
- Documents split into overlapping chunks to preserve context across boundaries
- Each chunk embedded into a vector representation
- Query embedded the same way and matched via similarity search
- Top-k chunks injected into the prompt as grounding context
- FastAPI endpoint exposes the pipeline for integration into other apps
Tech stack
Challenges & solutions
- Chunking strategy — too small and context is lost, too large and retrieval precision drops. Resolved with overlapping chunks sized to keep single ideas intact.
- Irrelevant retrieval — early versions occasionally pulled loosely related chunks. Addressed by tuning similarity thresholds and top-k values.
- API integration — designed the pipeline behind a FastAPI endpoint from the start so it could be dropped into other applications, not just used as a standalone script.
Results
Future improvements
- Add a reranking step after initial retrieval to further improve answer precision
- Support multi-document source citation in the response itself
- Add a lightweight eval harness to track answer groundedness over time