Grounded answers, not hallucinations
GenAI Smart Medical Chatbot
RAG chatbot that answers medical questions grounded in real documents, with multi-turn memory.
- Embedding dims
- 384
- Retrieval
- Top-k semantic
- Memory
- SQLite, multi-turn
the problem
A raw LLM asked a medical question will answer confidently whether or not it knows. In a medical context that is actively dangerous. The chatbot needed to ground every answer in actual medical documents and hold context across a multi-turn conversation, because health questions rarely come one at a time.
the solution
A RAG system orchestrated with LangChain: documents get chunked, embedded with 384-dimension HuggingFace sentence embeddings, and indexed in Pinecone. At question time, top-k semantic search retrieves the most relevant chunks and Llama 3.1 answers strictly from that context. Conversation history is persisted in SQLite, so follow-up questions ('what about the dosage?') resolve against everything said before. The whole thing is served through Flask and deployed live.
tech stack
- LangChainRAG orchestration: chunking, retrieval, prompt assembly
- Llama 3.1Answer generation grounded in retrieved context
- PineconeVector index for top-k semantic search
- HuggingFace384-dim sentence embeddings
- SQLitePersistent multi-turn conversation memory
- FlaskAPI and chat interface
key engineering decisions
384-dim embeddings over larger models
Smaller sentence embeddings kept indexing and retrieval fast and cheap. For chunk-level retrieval over a bounded medical corpus, the accuracy difference against larger embedding models did not justify the cost.
SQLite for conversation memory
Conversation state needs durability, not scale. SQLite gave persistent, queryable memory with zero infrastructure, which kept the deploy simple enough to run on a free-tier host.
Retrieval-first prompting
The prompt template instructs the model to answer only from retrieved chunks. Combined with top-k retrieval tuning, this is the main defense against the model inventing medical facts.