RAG Chat Engine: Powering the Portfolio AI Chatbot with Zero External Vector DB

Why This Portfolio Has a Real AI Chatbot

Most portfolio chatbots are fake — a scripted FAQ in a chat bubble. This portfolio's chatbot is a real production RAG pipeline: it embeds a knowledge corpus (my resume, projects, blog posts), retrieves semantically relevant chunks on each query using cosine similarity, and generates grounded answers with Gemini.

Every answer the chatbot gives is traceable to a specific chunk of real content. If you ask "What is Career Agents?" it retrieves the Career-Agents chunks and generates an answer grounded in the actual project description — not a hallucinated response.

The Full Pipeline

Corpus (MDX/text files)
    ↓ corpus builder
Text Chunks (400-token segments with 80-token overlap)
    ↓ Gemini text-embedding-004
Embedding Vectors (768-dim per chunk)
    ↓ saved to embeddings.json
    
Query
    ↓ Gemini text-embedding-004
Query Vector (768-dim)
    ↓ cosine similarity against all corpus vectors
Top-K Chunks (k=5 by default)
    ↓ assembled as context
Gemini Flash
    ↓
Grounded Answer

Step 1: Corpus Building

// scripts/build-corpus.ts
import { readFileSync, readdirSync } from "fs";
import matter from "gray-matter";

interface CorpusChunk {
  id: string;
  content: string;
  source: string;
  metadata: Record<string, unknown>;
}

function buildCorpus(): CorpusChunk[] {
  const chunks: CorpusChunk[] = [];
  
  // Index blog posts
  const blogFiles = readdirSync("content/blog").filter(f => f.endsWith(".mdx"));
  for (const file of blogFiles) {
    const raw = readFileSync(`content/blog/${file}`, "utf-8");
    const { data, content } = matter(raw);
    
    // Split into chunks
    const segments = chunkText(content, { chunkSize: 400, overlap: 80 });
    segments.forEach((segment, i) => {
      chunks.push({
        id: `blog:${file}:${i}`,
        content: segment,
        source: `blog/${file}`,
        metadata: { title: data.title, date: data.date, tags: data.tags },
      });
    });
  }
  
  // Index profile data (skills, experience, achievements)
  const profile = require("../src/data/profile");
  chunks.push({
    id: "profile:summary",
    content: profile.profile.summary,
    source: "profile/summary",
    metadata: { type: "profile" },
  });
  
  // Index projects
  const { projects } = require("../src/data/projects");
  for (const project of projects) {
    chunks.push({
      id: `project:${project.slug}`,
      content: `${project.name}: ${project.overview} Stack: ${project.stack.join(", ")}`,
      source: `projects/${project.slug}`,
      metadata: { name: project.name, category: project.category },
    });
  }
  
  return chunks;
}

Step 2: Embedding the Corpus

// scripts/build-embeddings.ts
import { GoogleGenerativeAI } from "@google/generative-ai";
import { writeFileSync } from "fs";

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);
const embeddingModel = genAI.getGenerativeModel({ model: "text-embedding-004" });

interface EmbeddingEntry {
  id: string;
  content: string;
  source: string;
  metadata: Record<string, unknown>;
  embedding: number[];
}

async function buildEmbeddings(): Promise<void> {
  const corpus = buildCorpus();
  const entries: EmbeddingEntry[] = [];
  
  // Process in batches to respect rate limits
  const BATCH_SIZE = 10;
  for (let i = 0; i < corpus.length; i += BATCH_SIZE) {
    const batch = corpus.slice(i, i + BATCH_SIZE);
    
    const embeddings = await Promise.all(
      batch.map(chunk =>
        embeddingModel.embedContent({
          content: { parts: [{ text: chunk.content }], role: "user" },
          taskType: "RETRIEVAL_DOCUMENT",
        })
      )
    );
    
    batch.forEach((chunk, j) => {
      entries.push({
        ...chunk,
        embedding: embeddings[j].embedding.values,
      });
    });
    
    console.log(`Processed ${Math.min(i + BATCH_SIZE, corpus.length)}/${corpus.length} chunks`);
    await sleep(200);  // rate limit buffer
  }
  
  writeFileSync("src/data/embeddings.json", JSON.stringify(entries, null, 2));
  console.log(`Built ${entries.length} embeddings`);
}

Using taskType: "RETRIEVAL_DOCUMENT" at index time and taskType: "RETRIEVAL_QUERY" at query time is a Gemini embedding API requirement — different task types produce embeddings optimized for their role.

Step 3: Query-Time Retrieval

// src/lib/rag.ts
import embeddingsData from "../data/embeddings.json";

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0, normA = 0, normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

export async function retrieveChunks(
  query: string,
  k: number = 5,
): Promise<RetrievedChunk[]> {
  const embeddingModel = genAI.getGenerativeModel({ model: "text-embedding-004" });
  
  const queryEmbedding = await embeddingModel.embedContent({
    content: { parts: [{ text: query }], role: "user" },
    taskType: "RETRIEVAL_QUERY",
  });
  
  const queryVector = queryEmbedding.embedding.values;
  
  // Score all chunks
  const scored = embeddingsData.map(entry => ({
    ...entry,
    similarity: cosineSimilarity(queryVector, entry.embedding),
  }));
  
  // Return top-k
  return scored
    .sort((a, b) => b.similarity - a.similarity)
    .slice(0, k)
    .filter(chunk => chunk.similarity > 0.6);  // minimum threshold
}

Step 4: Grounded Generation

// src/app/api/chat/route.ts
export async function POST(req: Request) {
  const { message, history } = await req.json();
  
  // Retrieve relevant chunks
  const chunks = await retrieveChunks(message, 5);
  
  if (chunks.length === 0) {
    return Response.json({ 
      reply: "I don't have specific information about that in my knowledge base. Try asking about my projects, skills, or experience." 
    });
  }
  
  // Build grounded context
  const context = chunks
    .map(c => `[Source: ${c.source}]\n${c.content}`)
    .join("\n\n");
  
  const systemPrompt = `
    You are an AI assistant for Karthik Rajesh Shet's portfolio.
    Answer questions using ONLY the provided context.
    If the context doesn't contain the answer, say so honestly.
    Be conversational but precise. Cite sources where relevant.
    
    Context:
    ${context}
  `;
  
  const gemini = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
  const chat = gemini.startChat({
    history: history.map((m: { role: string; content: string }) => ({
      role: m.role,
      parts: [{ text: m.content }],
    })),
    systemInstruction: systemPrompt,
  });
  
  const result = await chat.sendMessage(message);
  return Response.json({ reply: result.response.text() });
}

Why No External Vector Database?

For a portfolio site with a few hundred corpus chunks, a vector database is unnecessary overhead. The embeddings.json file (a few MB) loads at startup and cosine similarity search across 200-300 chunks completes in under 5ms — faster than any database round-trip.

At scale (10k+ chunks), you'd switch to pgvector or Pinecone. For this use case, JSON is perfect.

Rebuilding Embeddings

When content changes (new blog post, updated project), run:

npm run build:embeddings

This re-embeds only changed chunks (tracked by content hash) and updates embeddings.json. The re-embedding takes about 30 seconds for a full rebuild of 300 chunks.