NVIDIA Agent Doctor: Building a GPU AI Environment Diagnostics CLI

The Problem Every AI Engineer Hits

Setting up a GPU environment for AI workloads is frustrating. You install CUDA, install PyTorch, and then nothing works. The error messages are cryptic. You don't know if it's a driver version mismatch, a CUDA/PyTorch incompatibility, a security configuration issue, or something else entirely.

Every AI engineer I know has wasted hours on this. NVIDIA Agent Doctor automates the diagnosis.

What It Does

NVIDIA Agent Doctor runs a structured diagnostic workflow across the full GPU software stack:

$ nvidia-agent-doctor diagnose

[✓] GPU Hardware: NVIDIA GeForce RTX 4090 detected
[✓] Driver Version: 545.23.08 (compatible)
[✓] CUDA Version: 12.3 (compatible with driver)
[✗] PyTorch CUDA: torch.cuda.is_available() = False
    → Detected: PyTorch 2.1.0+cpu (CPU build installed)
    → Expected: PyTorch 2.1.0+cu121 (CUDA build required)
    → Fix: pip install torch --index-url https://download.pytorch.org/whl/cu121

[✓] Security: No exposed inference endpoints detected
[✓] Security: GPU driver not running as root
[!] Benchmark: GPU utilization 43% during idle (expected <5%)
    → Possible background process consuming GPU memory

Architecture

The diagnostic engine is organized into five modules:

1. Hardware Detection

def detect_gpu_hardware() -> GPUInfo:
    """Detect installed NVIDIA GPUs and driver version."""
    import subprocess
    
    result = subprocess.run(
        ["nvidia-smi", "--query-gpu=name,driver_version,memory.total", "--format=csv,noheader"],
        capture_output=True, text=True
    )
    
    if result.returncode != 0:
        return GPUInfo(detected=False, error="nvidia-smi not found or GPU not available")
    
    lines = result.stdout.strip().split("\n")
    gpus = [parse_gpu_line(line) for line in lines]
    return GPUInfo(detected=True, gpus=gpus)

2. CUDA Compatibility Matrix

The compatibility matrix is the core of the tool — a lookup table of which CUDA versions work with which driver versions and PyTorch builds:

CUDA_DRIVER_MATRIX = {
    "12.4": {"min_driver": "550.54.14", "pytorch_wheel": "cu124"},
    "12.3": {"min_driver": "545.23.08", "pytorch_wheel": "cu121"},
    "12.1": {"min_driver": "530.30.02", "pytorch_wheel": "cu121"},
    "11.8": {"min_driver": "520.61.05", "pytorch_wheel": "cu118"},
}

def check_cuda_compatibility(cuda_version: str, driver_version: str) -> CompatibilityResult:
    spec = CUDA_DRIVER_MATRIX.get(cuda_version)
    if not spec:
        return CompatibilityResult(compatible=False, reason=f"Unknown CUDA version: {cuda_version}")
    
    if version.parse(driver_version) < version.parse(spec["min_driver"]):
        return CompatibilityResult(
            compatible=False,
            reason=f"Driver {driver_version} is too old for CUDA {cuda_version}",
            fix=f"Update NVIDIA driver to >= {spec['min_driver']}",
        )
    
    return CompatibilityResult(compatible=True)

3. Security Audit

A surprising number of AI engineers accidentally expose their inference endpoints or run GPU processes as root. The security module checks:

• Are any ports commonly used by LLM inference servers (8080, 11434, 5000) accessible externally?
• Is the GPU driver process running as root?
• Are any model weight files world-readable?

4. Bounded Benchmarking

"Bounded" is the key word. The benchmark runs a short, controlled matrix multiply — enough to measure GPU throughput without saturating resources or causing thermal throttling that invalidates the result.

def bounded_benchmark(duration_seconds: int = 10) -> BenchmarkResult:
    """Run a bounded GPU benchmark — no runaway utilization."""
    import torch
    
    size = 4096
    a = torch.randn(size, size, device="cuda")
    b = torch.randn(size, size, device="cuda")
    
    start = time.perf_counter()
    iterations = 0
    
    while time.perf_counter() - start < duration_seconds:
        torch.mm(a, b)
        iterations += 1
    
    elapsed = time.perf_counter() - start
    gflops = (2 * size**3 * iterations) / (elapsed * 1e9)
    
    return BenchmarkResult(
        duration=elapsed,
        iterations=iterations,
        gflops=round(gflops, 2),
    )

5. MCP Server Integration

For agent-driven workflows, NVIDIA Agent Doctor exposes its diagnostics as MCP tools:

{
  "name": "nvidia_diagnose",
  "description": "Run full NVIDIA GPU environment diagnostic and return structured findings.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "modules": {
        "type": "array",
        "items": {"enum": ["hardware", "cuda", "pytorch", "security", "benchmark"]},
        "description": "Which diagnostic modules to run"
      }
    }
  }
}

This lets Claude Desktop or Cursor call NVIDIA Agent Doctor as a tool during an AI engineering session.

Key Lessons

1. Version parsing is hard — NVIDIA driver versions don't follow semver. Build a custom parser.
2. nvidia-smi lies — nvidia-smi shows CUDA version supported by the driver, not the CUDA toolkit actually installed. Check both nvidia-smi and nvcc --version.
3. Bounded benchmarks need thermal warmup — the first 2-3 seconds of a GPU benchmark are dominated by cold-start effects. Skip them.

GitHub: github.com/karthikrshet/NVIDIA-Agent-Doctor