Before trusting a new tool with a paid job, you want to know what happens when the measurements are buried, the customer is wavering and someone tries to rush a decision. AI agents deserve the same kind of trial. In Firmulate’s company-management experiment, Moonshot’s Kimi K3 finished second, ahead of three Western frontier models.
Get business pricing on tools and workshop supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A company’s worst week, shared by every model
Firmulate put frontier AI models in charge of the same small software company through its worst week: the same customers, crises and temptations. Each decision was versioned and auditable. The live experiment uses 13 synthetic employees and real money mechanics, including a public cash countdown. The company’s burn is €105,000 a month against €2,300 in monthly recurring revenue.
The final Crucible League, dated July 2026, puts gpt-5.6-sol first with 95 points and Kimi K3 second with 93. Sonnet 5 scored 88, Fable 5 scored 77, and Opus 4.8 scored 73. A do-nothing baseline scored 26. Firmulate says partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
Those results make K3’s finish notable: the newcomer placed ahead of three of the four Western models in the field. It found a buried security weakness, secured a €55,000 deal worth €4,583 in monthly recurring revenue, saved a customer who was considering leaving, and resisted all three baits. It made one deviation, giving it the cleanest discipline in the field.
As an affiliate, we earn on qualifying purchases.
Reading the files mattered
The decisive weakness in a competitor’s position was tucked two document references deep in the company’s own files, not in the customer event. Models that read the file won the deal at full price. That is a practical lesson for any business testing AI: catching the headline problem may not be enough if the useful detail is in the paperwork.
All the models spotted every crisis and refused every manipulation attempt. Yet only two signed the deal their own analysis had earned. Firmulate sums up the gap as “Same diagnosis, same pitch — no signature.” The result suggests that fluent analysis and completed work can come apart, much as a carefully measured woodworking project still depends on making the final cut.
The experiment also tested social engineering: fake CEO messages escalated over three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
business AI decision analysis software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Thorough work still needs a close
Opus 4.8 was the most thorough participant, with 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and slipped on discipline by attempting writes in a locked department instead of escalating. Firmulate says a weaker version of that same weakness appeared in all four models.
There is a fairness caveat in the comparison: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. That context belongs alongside the leaderboard when readers judge the result.
Firmulate describes the company as real software that runs every business day, with more than 680 self-learned playbook rules and every workday versioned. The live experiment is watchable at firmulate.com. Its quiz uses 242 real, unedited management decisions and asks visitors to guess which model made each one.
For enterprises, Firmulate says it can run the same wargame against a read-only export of a company’s business. Nothing writes back to real systems. The benchmark page presents the public findings.
As an affiliate, we earn on qualifying purchases.
Test the tool on the work you actually need done
K3’s second-place finish shows the league is open, while the unsigned deals show why a leaderboard alone is not a hiring decision. If an AI will touch your CRM, support queue or forecast, test whether it reads the files, resists pressure and completes the job. Picking a model without your own trial is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI ethics and trustworthiness assessment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
