The Agent Buyer's Map
Ten dimensions every executive should score before they buy, lease, or deploy an AI agent. No Magic Quadrant, no analyst rubric, no vendor spin — one scorecard, one vocabulary, the same whether it's a service agent, a marketing workflow, or an internal copilot.
There is now a marketplace category that didn't exist eighteen months ago, with no shared buyer vocabulary, no Gartner Magic Quadrant, no Forrester Wave, no procurement template, and — in most cases — no internal owner senior enough to be accountable for the decision. That category is AI agents. And the people writing checks for them right now are doing it blind.
This is the asymmetry the vendor ecosystem is feeding on. Every platform that ships an agent ships its own evaluation framework with it — and every one of those frameworks is structured to make their agent look good. Salesforce's criteria flatter Agentforce. Microsoft's flatter Copilot. HubSpot's flatter Breeze. The buyer ends up trying to compare three vendors using three different vocabularies, none of which were designed to help them choose.
Built to win the demo
Ships with the agent. Scores the dimensions the vendor is strong on, in the vendor's words. Three vendors, three vocabularies — none built to help you choose.
Built to survive month four
Independent and buyer-side. Ten dimensions, one number, the same language across every vendor and use case. The framework the vendors hope you don't read before the demo.
Why "AI evaluation" isn't enough
Most agent-buying advice sits at the wrong altitude.
Model benchmarks (MMLU, GPQA) tell you which underlying LLM is smarter — but you're not buying a model, you're buying an agent built on top of one. Security questionnaires (SOC 2, ISO 27001, data residency) tell you the agent won't leak — but say nothing about whether it will work. SaaS-style RFPs treat the agent as a static product — but agents drift, learn, and accumulate context in ways SaaS does not.
The Buyer's Map fills the gap in between. It assumes the agent is technically functional and security-compliant, and scores the dimensions that decide whether the deployment actually delivers value at twelve months — or quietly fails at month four, when the novelty wears off and operational reality kicks in. Each dimension is scored 0–5. Below 30, you're not ready to buy this agent — or it isn't ready for you. 40 or above, the deployment is likely to compound rather than decay.
The scorecard · score it live
Score the agent in front of you.
Set each dimension 0–5 as you read — the total and the verdict update live. The detailed rubric for every dimension, with what 0, 3, and 5 actually look like, sits just below. When you're done, print the one-pager and take it into the room.
The scorecard is the easy part. The hard part is who scores it — and the rubric below is what keeps the scores honest. Score the agent before the demo, again after the pilot, again at month six. When the scores diverge from the vendor's promises, you've just bought yourself the most important data point in the deployment.
The rubric
The ten dimensions — and what 0, 3, and 5 look like.
Five of these are auxfirst frameworks you can already go deep on; the deep-dive is linked on each. The Map is the buyer-side view of work we usually do from the builder's seat.
Can you describe the agent's job in one sentence — narrow enough to be falsifiable?
The number-one cause of failed deployments isn't model quality. It's job sprawl. "An agent that helps with customer service" fails. "An agent that resolves password-reset tickets without human handoff in under 60 seconds" succeeds — because it can be measured, scoped, and audited. If the pitch deck needs three sentences to define the job, score low. If it needs a paragraph, score zero.
Which stage of trust is this agent sold for — and does it match the stage you're ready to extend?
Drawn from the auxfirst trust architecture — Functional → Contextual → Judgment → Advocacy. Most agents on the market operate at Functional trust (a bounded task on command). Many are sold as if they operate at Judgment trust (good calls in ambiguous situations). That mismatch is where money disappears. Score by comparing what the vendor claims against what they can demonstrate with named references at month nine.
What does this agent need from your stack to work — and how is that disclosed?
Every demo runs in a sandbox where the data is clean, the APIs respond, and permissions are pre-granted. Every deployment runs in your stack, where they aren't. Ask for the full list: API endpoints, auth scopes, data-residency assumptions, write permissions, rollback paths. Arrives in one document on the first request — score high. Arrives in fragments over four follow-up calls — score low.
If this agent makes a wrong call at scale, what's the cost — and how fast can you stop it?
Agents act. SaaS records. This is the categorical difference, and the one procurement under-weights most. An agent that auto-issues refunds has a different blast radius than one that drafts refunds for approval — and the price difference between them rarely reflects the risk difference. Score on three sub-questions: maximum exposure per wrong action; how fast a human can override or kill it; whether the audit trail can reconstruct the decision.
Where does the agent's memory of your organization live — and what are the rules?
Every serious agent accumulates memory: your customers, processes, past decisions, brand voice, edge cases. That memory is now a brand asset and a compliance surface. Where does it physically live, who has write access, what's the retention policy, how is it audited, what happens to it if you switch vendors? If the vendor can't answer in writing, the agent isn't ready for procurement — however impressive the demo.
How does this agent fail — and how willing is the vendor to talk about it?
Vendors love to demo the happy path. Score what happens off it. Confident wrong answers are one failure mode; silent capability degradation as the base model is updated is another; cascading errors when a downstream system returns unexpected data is a third. The diagnostic question: "Show me your three most embarrassing failure modes from last quarter and what you did about them." The quality of that answer is the score.
Who, inside your organization, actually runs this agent day-to-day?
Every agent creates a new role: someone monitors it, corrects it, tunes its prompts, manages edge cases, explains its outputs to the business. This role is almost never staffed at procurement time, and it almost always decides whether the deployment succeeds. "Anyone can use it" — score zero. "A dedicated ops analyst, ~10 hours a week on monitoring, tuning, and exceptions, with these tools" — score five.
Can you defend this purchase to legal, compliance, finance, and regulators — using the vendor's artifacts?
This is the dimension that most determines whether the agent makes it past pilot. The tech can work and the ROI can be real, but if you can't pass internal review, it stays in the sandbox. For B2C: consumer disclosure, claim substantiation, fairness audits. For FMCG: per-market regulatory defensibility. For financial services: model-risk documentation. For Europe: EU AI Act classification and conformity-assessment readiness. If the vendor can't produce a procurement-ready package mapped to your jurisdiction on day one, it isn't enterprise-ready.
If this agent doesn't work out in twelve months, what does leaving cost — in money, time, and context?
The marketplace quietly assumes agents are interchangeable. They aren't. The moment one accumulates memory, edge-case handling, prompt refinements, and integration plumbing, switching gets expensive — and most vendors design for that lock-in without naming it. The fair version of this dimension isn't zero switching cost. It's transparent switching cost: a vendor who hands you, in writing, exactly what you'd lose and keep when leaving is treating you as an adult. One who deflects is signalling something.
Will this agent be more valuable in month twelve than in month one — and what is the mechanism?
The SaaS comparison breaks here. A static SaaS tool delivers the same value on day 365 as on day 1. An agent should not — the whole point of memory, judgment, and operator-driven refinement is that it improves with use. If it doesn't, you're paying agent prices for SaaS value. Score the mechanism: is there a feedback loop from operator corrections back into behavior? Is memory accumulating into a defensible context layer? Are there explicit month-3/6/12 milestones? This is the dimension that separates an agent purchase from a chatbot purchase.
How to use the Map
The scorecard is the easy part. The hard part is who scores it.
The AI champion who introduced the vendor.
They've already invested social capital in the answer. They will, sincerely and without realizing it, anchor every score upward.
A small cross-functional panel.
The operator who'll run it, someone from compliance or legal, a finance owner, and a senior sponsor who is not the AI champion. Score independently, then reconcile.
Disagreements are diagnostic — they almost always point to the dimensions the deployment will fail on later. A score of 30 or below means the agent is not procurement-ready or your organization is not deployment-ready. Both are valuable findings, and usually it's some of each. The next move isn't to push harder on procurement — it's to fix the readiness gap on whichever side it sits.
Why we're publishing this
The category needs a shared buyer vocabulary more than we need to gatekeep one.
We do this work for a living. auxfirst's audit and sprint engagements use a more detailed version of this exact framework, calibrated to industry and use case, run as a facilitated panel rather than a solo scorecard. We could have kept it private. We're publishing it because a blind buying market is bad for everyone — including the good vendors. The Map is part of the auxfirst open vocabulary for agentic systems, alongside the Trust Canvas and TrustKit, the AUX Heuristics, and the four-stage Trust Architecture. CC BY 4.0 — no "request a demo" wall, no analyst gatekeeping.
If you're an executive about to write a check, the Map is yours. Score it above, print it, take it into the room. If you'd rather have us run the panel — bringing industry-calibrated weights, vendor-side benchmarking from our agent-readiness database, and the procurement-defense documentation — that's the Buyer's Audit: one facilitated panel, two weeks, a defensible recommendation at the end. But you don't need us to start. The Map starts working the moment you score the first dimension.
"The agent was cleared to buy" is a thin defence for a deployment nobody scored.
The agents that compound and the agents that quietly fail at month four look identical in the demo. They diverge on the ten dimensions a vendor framework was never built to surface — job clarity, trust-stage honesty, blast radius, memory, failure candour, the operator, defensibility, switching cost, and the compounding mechanism.
Score the agent before the demo, again after the pilot, again at month six. If the numbers drift from the promises, that gap is the most valuable thing you'll learn in the whole procurement.
Score it before you sign. The Map is yours.
Want the panel run for you? That's the Buyer's Audit.
Industry-calibrated weights, vendor-side benchmarks from our agent-readiness database, and a procurement-defense pack — one facilitated panel, two weeks, a defensible recommendation at the end.
Book a Buyer's Audit →