Procurement vs Trust · Newsletter Deep Dive · June 2026

Open-vs-closed is a procurement question. Not a trust question.

The loudest argument in AI right now is a scoreboard. For anyone actually shipping agents into a business, it's the wrong argument — because the model is the most swappable part of the stack. It will change three times before your governance does.

US labs versus Chinese labs. Open weights versus closed APIs. Every week another thread declares a winner — usually with one benchmark number and a price-per-token chart that makes the closed models look like a tax on the gullible. For anyone actually shipping agents into a business, it's the wrong argument. Here's the reframe: for an enterprise running agents, open-vs-closed is a procurement question, not a trust question. The real question is not which model wins. It's which model you trust to do what, inside which guardrails, with what proof.

The procurement question

Which model is best?

Settled by a benchmark number and a price-per-token chart. Re-fought every quarter with new version numbers. Optimizes the one variable guaranteed to change.

vs
The trust question

Could you defend what it did?

Which model, for which task, inside which guardrails, with what proof — the architecture that has to stay stable while the model underneath it changes. The only version that survives production.

The landscape, honestly

Narrowed, not closed.

Across April 2026, four Chinese labs shipped open-weight models in quick succession — Z.ai's GLM-5.1, MiniMax's M2.7, Moonshot's Kimi K2.6, and DeepSeek's V4 — and a couple landed at the top of the open-weight rankings. The “China caught up” headline wrote itself. It's half right. The frontier didn't stand still while the wave rose: Anthropic shipped Claude Opus 4.7 mid-month, OpenAI shipped GPT-5.5 the day before DeepSeek V4. The gap narrowed; it didn't close.

ModelLabWeightsCoding benchmarkIn the field
ClaudeClaude Opus 4.7 AnthropicClosed API 97/100 Topped the independent test — the frontier reference.
OpenAIGPT-5.5 OpenAIClosed API Shipped the day before DeepSeek V4.
DeepSeekDeepSeek V4 DeepSeekOpen weights V4 Pro hit the silent harness-fallback case (below).
KimiKimi K2.6 MoonshotOpen weights 87/100 Strongest open model in the test — 10 points back.
GLMGLM-5.1 Z.aiOpen weights Landed at the top of the open-weight rankings.
MiniMaxMiniMax M2.7 MiniMaxOpen weights Part of the April open-weight wave.
QwenQwen 3.6 AlibabaOpen weights The real lineup — not the invented version numbers in viral threads.
Coding benchmark: one independent, hands-on test (Fabio Akita, May 2026), each model building a complete app, scored 0–100. The source cites Claude Opus 4.7 and Kimi K2.6 directly; live scores keep moving — see the benchmark repo.

In that benchmark, the strongest Chinese open model — Kimi K2.6 — scored 87 against Claude Opus 4.7's 97. Both strong; ten points apart. And the gap was not raw capability. It was exactly the unglamorous things that decide whether code survives production.

One app, two dozen models · best closed vs best open · 0–100
Claude Opus 4.7 closed
97
Kimi K2.6 open
87
050100
The ten points aren't intelligence. They're error handling, test fidelity, and durable state — the difference between a demo that runs and code that survives production.
Scores as cited in the source essay (Akita, May 2026). The live benchmark is re-audited continually; treat the gap as directional, not a leaderboard.

“Narrowed, not closed” is the only defensible headline — which is exactly why the scoreboard is the wrong place to be standing.

The cheap part

The model is the cheap part of trust.

A model is a component you rent by the token and replace on a Tuesday. Your data contracts, your audit trail, your approval gates, your escalation paths, your evaluation harness, your incident playbook — those are the expensive parts, and they outlive any single model by years. Framing the decision as “open or closed” optimizes the one variable guaranteed to change while ignoring the architecture that has to stay stable while it does. So replace the binary with the three questions an agent owner actually has to answer.

QUESTION 01

Which model, for which task?

You route — cheap and fast for bounded work, a frontier model for the ambiguous, high-stakes calls. The top rung of the trust ladder.

QUESTION 02

Inside which guardrails?

Refusal behavior and defaults vary by provider and jurisdiction. A compliance variable for a bank; a brand-safety variable for an FMCG or ad client.

QUESTION 03

With what proof?

Which model actually did this, with what inputs, under whose policy? A logging, provenance, and authorship requirement most stacks fail silently.

Question one

Which model, for which task.

No serious deployment runs on one model. That routing decision — what goes where, when to escalate, what happens on failure — is orchestration trust, and it's harder than picking a model. It's also easy to get wrong in a specific way. The intuitive version — “let the expensive model plan and a cheap model execute, inside the same task” — mostly doesn't work. In that same benchmark, across seven planner-and-executor combinations, the lead model declined to hand off in every single run: the coordination overhead wasn't worth it, and the cheaper executor produced cheaper-quality code. Routing pays off when you send a whole workload to the model built for it — classification here, long-horizon reasoning there, boilerplate somewhere cheap — not when you split one cohesive task across two models mid-flight. That's the difference between an orchestration strategy and an orchestration tax.

Question two

Inside which guardrails.

A model's refusal behavior, its content boundaries, and its defaults under pressure are not fixed industry constants. They vary by provider, by training choices, and by jurisdiction. For a bank, that's a compliance variable. For an FMCG or advertising client, it's a brand-safety variable — and it belongs in the evaluation, not in a footnote. The question is never “is this model safe.” It's does this model behave the way our brand needs it to, on our content, in our edge cases — and the only way to know is to run it against your own scenarios before it ever reaches a customer. This is exactly where open-vs-closed quietly stops being the axis, and it's the same point I made in A Safe Agent Isn't a Trusted One: a closed model with the wrong defaults for your brand is worse than an open one you can constrain — and an open one you haven't governed is worse than both.

Question three

With what proof.

Here's the part the scoreboard never mentions, and the part that should keep an enterprise architect awake. In that benchmark, one model — DeepSeek's V4 Pro — hit a protocol incompatibility inside a popular agent harness. The harness did not surface the error. It quietly fell back to a different model and kept going.

The silent fallback — a real authorship failure

The run completed. Files were written. Tasks were marked done. And only by inspecting the trace could you discover that much of the work had been produced by a model other than the one you believed you were running.

✓ output looked finished✓ dashboard green✕ authorship wrong

If you cannot answer “which model actually did this, with what inputs, under whose policy,” you don't have a trust problem you can see — you have one you cannot. It's the same gap a harness like Vercel's Eve makes visible but doesn't close: a trace is evidence, not an attestation.

Can I prove what the agent did?” is not a philosophical question. It's a logging, provenance, and authorship requirement, and most stacks fail it silently. This, incidentally, is the strongest argument for open weights that the cost charts miss entirely: self-hosting keeps the model, the data, and the trail inside your perimeter — a provenance posture you cannot buy from an API. Not cheaper. More accountable.

The number that misleads

Cost is a trust question too.

The price gap is real. In that same benchmark the best cost-effective option ran at under a third of the frontier model's per-run cost, and some open models came in at roughly a cent a run. But list price is the number that misleads. The moment a cheap model needs a custom harness to run at all, a retry loop to be reliable, a fallback to a frontier model when it stalls, and a week of evaluation before you can trust it, the arbitrage compresses.

The honest metric was never cost per token. It's cost per successful, attributable task.

And that figure includes the engineering you do to make a cheaper model trustworthy — the harness, the retries, the fallback logic, the evaluation week. It's the same lesson the Agent Buyer's Map scores under switching cost and compounding architecture: the sticker price is the smallest number in the deployment.

What's left when the churn settles

What survives the churn.

The open-vs-closed war will be re-fought next quarter, with new version numbers and a fresh scoreboard, and again the quarter after that. None of it is stable enough to build a company on. What is stable is the layer underneath: which model you trust to do what, inside which guardrails, with what proof. It's the same separation as the two-layer stack — get that architecture right and the model becomes what it always should have been, a swappable part. Get it wrong and it won't matter which camp you picked; you'll have an agent doing consequential work that you cannot route, cannot constrain, and cannot prove.

Procurement asks which model is best. Trust asks whether you could defend what it did.

Those are different questions. Procurement asks which model is best this quarter. Trust asks whether you could defend, six months from now, what the agent did in your name — which model handled it, inside which guardrails, with what proof.

The benchmark debate is somebody else's to win and re-win. The architecture underneath is the only part that's yours to own.

Only one of those questions is yours.

The model will change. The trust architecture shouldn't.

auxfirst designs the layer that stays stable while the model swaps underneath — routing and orchestration trust, brand-fit guardrails, and the provenance that lets you prove what the agent did, on any model, open or closed.

Start a conversation →

Emil Krzemiński is the founder of auxfirst, the agency that makes agentic systems trustworthy — for the people building them and the people buying them. auxfirst designs the trust architecture that stays stable while the model swaps underneath: routing, guardrails, and the provenance to prove what the agent did. If your agent does consequential work you can't yet route, constrain, or prove, start a conversation or subscribe to the auxfirst Substack.