RAG Chat Engine
Production-grade Retrieval-Augmented Generation API with vector search and Gemini
Overview
A full Retrieval-Augmented Generation pipeline that embeds a custom knowledge corpus into vector space, retrieves semantically relevant chunks on each query, and feeds them as context to Gemini AI — grounding every response in real source documents rather than hallucinated knowledge.
Problem
General-purpose LLMs confidently hallucinate when asked domain-specific questions. RAG systems ground responses in verified source documents, but building a production-quality pipeline requires non-trivial work on chunking, embedding, retrieval, and context assembly.
Solution
A corpus builder that chunks source documents by section, a Gemini-powered embedding pipeline that converts chunks to vectors, a cosine-similarity retrieval layer, and a chat API that assembles retrieved chunks into a grounded context window before calling Gemini for generation.
Architecture
- 1Corpus builder parses MDX/text documents into semantically meaningful chunks
- 2Embedding pipeline calls Gemini text-embedding-004 to convert each chunk to a 768-dim vector
- 3Embeddings persisted as JSON for zero-infrastructure local deployment
- 4Query-time: embed the user question, compute cosine similarity against all chunks, retrieve top-K
- 5Chat API assembles retrieved chunks as context and calls Gemini for grounded generation
- 6Next.js API route exposes the full pipeline as a streaming chat endpoint
Technical Challenges
- Chunking strategy significantly affects retrieval quality — too small loses context, too large dilutes relevance.
- Cosine similarity at scale requires efficient vector operations without a dedicated vector database.
- Balancing context window size (more chunks = more context = higher cost and latency) against response quality.
Results
- Powers the AI chat widget on this portfolio — every answer is grounded in actual portfolio content.
- Zero external vector database dependency — embeddings stored as JSON, retrieval in TypeScript.
- Demonstrates a complete RAG pipeline in under 200 lines of core logic.