Live Agent Test: exactly what runs
24 recorded base attempts (3 standard tasks, 8 agent configurations), plus recorded retries for failed configurations, one report. This page is the scope your $99 buys, in full.
The three tasks
Every site gets the same three tasks, in the same order. That is what makes one report comparable to another, and comparable to your own site a month later.
- Read and understand. The agent must reach your site and correctly state what you offer.
- Find the answer. The agent must locate the one fact a buyer needs: your pricing or plans, or the closest equivalent for your site type.
- Reach integration. The agent must find how to actually use you: docs, API reference, or the signup or contact path (up to, never through, account creation).
The eight configurations
Each configuration is one model running in one harness. We draw them per test from our roster: 15+ frontier models (Claude, GPT-5, Gemini, DeepSeek, Qwen, Kimi and more) and 10+ agent harnesses (Claude Code, Codex, Gemini CLI, OpenClaw and more) using the browser tools real agents use (browser-use, Playwright MCP, computer-use).
Guaranteed coverage floor in every test: at least 6 distinct models and at least 4 distinct harnesses.
Any configuration that fails gets one retry on a stronger model, recorded and included as an additional attempt, so the report can show whether a stronger model succeeds where a weaker one gave up.
Your report names every configuration that ran. It never lists one that did not.
Boundaries
Public pages only. We never log in, never solve CAPTCHAs, never submit forms, never complete purchases. Requests are politely paced. Site owners can opt out any time via support@agentcapable.dev.
What you receive
A report with, per configuration:
- Outcome: completed, partial, or failed at a named step.
- Time and token cost of every attempt.
- Prioritized fixes.
Recordings of every run are included as proof. See the exact format: view a specimen report.
Delivery and guarantee
Delivered to your checkout email within 24 hours of payment. Late: full refund. If we cannot produce a valid report for your site, you are refunded automatically. Refunds are handled by Paddle, our Merchant of Record. Full policy: /refunds.