Prompt Engineering for Reliable AI Agents: What I Learned Building Career Agents

Building Career Agents — a multi-agent system that handles resume tailoring, interview prep, LinkedIn optimization, and job tracking — taught me that prompt engineering is less about creativity and more about engineering discipline.

Here's what I actually do.

Start with the output schema, not the instructions

The most reliable way to get consistent output from an LLM is to define the output schema before writing the prompt. If your Resume Agent should return a structured object, define that structure first:

const resumeOutputSchema = z.object({
  tailoredBullets: z.array(z.string()),
  keywordsAdded: z.array(z.string()),
  sectionOrder: z.array(z.string()),
  atsScore: z.number().min(0).max(100),
  reasoning: z.string(),
});

Then write your prompt to produce exactly that shape. Validate every response against the schema. If validation fails, retry with an error message in the prompt: "Your previous response failed validation because: [error]. Please return valid JSON matching this schema: [schema]."

Temperature: lower than you think for structured tasks

For structured outputs (JSON, formatted data), I use temperature 0.1–0.3. The creativity dial is counterproductive when you need deterministic structure.

For creative tasks (writing a LinkedIn summary, drafting cover letter bullets), temperature 0.7–0.9 produces more varied and human-sounding output.

The mistake I see most often: using a high temperature for structured tasks and wondering why the JSON keeps breaking.

Chain-of-thought for complex reasoning, JSON for simple extraction

Some tasks benefit from chain-of-thought reasoning before producing output. For my Interview Agent generating mock interview questions, I prompt it to reason about the role requirements first:

Think step by step:
1. What are the core technical skills required for this role?
2. What behavioral competencies does this job description emphasize?
3. What gaps exist between the candidate's resume and these requirements?

Based on your analysis above, generate 10 interview questions in the following JSON format: [...]

The reasoning step produces better questions than going straight to output. But I strip the reasoning from what gets persisted — only the structured output is stored.

System prompts: role + constraints + output format

My system prompts follow a consistent three-part structure:

You are [ROLE]. Your job is [SPECIFIC TASK].

You MUST:
- [Hard constraint 1]
- [Hard constraint 2]

You MUST NOT:
- [Hard prohibition 1]

Output format: Return only valid JSON matching this schema: [SCHEMA]
Do not include markdown code blocks or any text outside the JSON object.

The "must not" section is as important as the "must" section. Explicitly prohibiting common failure modes (markdown wrappers, preamble text, explanations outside the JSON) dramatically reduces the frequency of parsing failures.

Building an evaluation harness

The biggest mistake in prompt engineering is not having a systematic way to test whether your prompts improved. For Career Agents, I maintain a small labeled dataset per agent:

resume_agent_eval_cases/
  input_1.json    ← profile + job description
  expected_1.json ← manually verified good output
  input_2.json
  expected_2.json
  ...

After every prompt change, I run all eval cases and compute a simple match score. The score doesn't need to be fancy — even checking that required fields are non-empty and ATS scores are in a reasonable range catches most regressions.

Without this, you're flying blind. A prompt that feels better to write doesn't necessarily produce better outputs.

The meta-lesson

Treat prompts like code. Version them, test them, review changes. The most expensive prompt mistakes are the ones you don't discover until a user hits an edge case in production.