RAG Chat Engine
Production-grade Retrieval-Augmented Generation API with vector search and Gemini
Overview
A full Retrieval-Augmented Generation pipeline that embeds a custom knowledge corpus into vector space, retrieves semantically relevant chunks on each query, and feeds them as context to Gemini AI — grounding every response in real source documents rather than hallucinated knowledge.
Problem
General-purpose LLMs hallucinate domain-specific answers. RAG systems ground responses in verified source documents, but building a production-quality pipeline requires non-trivial work on chunking, embedding, retrieval, and context assembly.
Solution
A corpus builder that chunks source documents by section, a Gemini-powered embedding pipeline converting chunks to vectors, a cosine-similarity retrieval layer, and a chat API that assembles retrieved chunks into a grounded context window.
Architecture
- 1Corpus builder parses MDX/text documents into semantically meaningful chunks
- 2Embedding pipeline calls Gemini text-embedding-004 for 768-dim vectors
- 3Embeddings persisted as JSON for zero-infrastructure local deployment
- 4Query-time: embed the user question, compute cosine similarity, retrieve top-K
- 5Chat API assembles retrieved chunks as context and calls Gemini for grounded generation
- 6Next.js API route exposes the full pipeline as a streaming chat endpoint
Technical Challenges
- Chunking strategy significantly affects retrieval quality.
- Cosine similarity at scale without a dedicated vector database.
- Balancing context window size against response quality and cost.
Results
- Powers the AI chat widget on this portfolio — every answer grounded in actual portfolio content.
- Zero external vector database dependency — embeddings stored as JSON.
- Complete RAG pipeline in under 200 lines of core logic.