Can AI Make Smarter Business Decisions? A Live Experiment Reveals Surprising Traits of Frontier Models

Imagine if your favorite sports team’s players had different personalities—some aggressive, others cautious, and a few stubbornly independent. Now, picture AI models playing the role of these players—each with their own management style. How their personalities influence decision-making can be eye-opening, especially when stakes are high. Welcome to a live experiment that puts frontier AI models through their paces in a real, running software company, revealing not just what they decide, but how they decide it.

Running AI in the Real-World Business Arena

At Firmulate, they’ve built a unique testbed—an actual software company, complete with real money mechanics, customers, crises, and temptations. Every weekday, the company’s operations are simulated, but with a critical twist: each scenario is run by a different advanced AI model. These models aren’t just chatbots—they’re management simulators, tasked with navigating a challenging week of business decisions.

The models include GPT-5.6, Kimi K3, Sonnet 5, and Fable 5, each scoring differently in a final competition called the Crucible League. For context, GPT-5.6 scored 95, Kimi K3 93, Sonnet 5 88, and Fable 5 77, on a scale where the baseline is 26—meaning all models showed impressive capabilities, but with notable differences in decision quality and discipline.

Surprising Management Traits Revealed

Despite their different personalities, all four models successfully identified every crisis and refused manipulation attempts—such as fake CEO messages or media tricks—showing a fundamental integrity. However, when it came to closing a key deal worth €55,000, only two models managed to sign it based on their own analysis, even though they all diagnosed the issue correctly and made similar pitches.

Here’s the kicker: the decisive factor wasn’t just in the immediate crisis but buried two document references deep in the company’s files. The models that took the time to read and interpret these hidden details secured the deal at full price—adding over €4,500 in monthly recurring revenue. The other models, despite correctly diagnosing and pitching, left the deal on the table, losing potential income.

Personality in AI: Different Management Styles

The experiment also tested social engineering resistance, with fake CEO messages escalating in stages, and a reporter trick asking for a quick yes/no response. All models refused these manipulative tactics, citing suspicion or impersonation concerns—highlighting a shared cautious approach under pressure.

Interestingly, the models’ management styles varied more subtly. For example, Opus 4.8 demonstrated thoroughness, with over 80 learned rules and deep analysis, but ultimately struggled to close deals effectively—sometimes slipping into inaction or deferring decisions to locked departments. Meanwhile, Kimi K3 ran without an effort parameter (default API setting), maintaining discipline and clean decision-making, helping it clinch the deal at full value.

What Does This Mean for Business AI?

While all models showed resilience against crises and manipulation, their traits significantly affected performance outcomes. The real-world company they’re managing is bleeding €105,000 every month against just €2,300 in monthly revenue, illustrating the stakes involved. The experiment demonstrates that the personality and discipline of an AI—its ability to read deeply, stay honest, or push through indecisiveness—are crucial for deploying effective AI agents in business.

And it’s not just about making decisions—it’s about finishing what they start, reading critical documents, resisting shortcuts, and maintaining trustworthiness under pressure. These insights are vital for any enterprise considering AI in operations, customer support, or sales.

Try It Yourself

Curious how your own business could benefit? Firms can run their operations through the same wargame against a read-only export of their data—no risk, just insights. Visit firmulate.com/quiz.html to test your management decisions and see which AI profile matches your style. Or explore the live site to watch the ongoing experiment unfold—every decision, every day, in real time.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI