Before an AI Runs the School, Put It Through a Worst Week

Age 18–24?Offer from Amazon

Prime made for students and young adults

• Fast, free delivery for dorm and study essentials
• Prime Video and Amazon Music included
• Member-only deals

Try Prime for Young Adults →Free trial for eligible 18–24 year olds

As an affiliate, we earn on qualifying purchases.Hide

Could an AI handle a school’s worst week?

Schools and education companies are weighing AI for more than drafting lesson plans. An agent might one day help manage customer support, forecast demand or respond to a crisis. The useful question is not only whether it can explain what to do. It is whether it will follow through, protect trust and stay within its authority when the pressure is on.

Firmulate is testing that question by putting AI models in charge of a live, synthetic company. Its final Crucible League offers a practical lesson for anyone considering AI in education: capable analysis does not guarantee sound action.

A shared test of judgment

In the experiment, each frontier model faced the same small software company during its worst week: the same customers, crises and temptations. Decisions were versioned and auditable. The point was to observe management behavior under pressure, not to judge a polished answer in isolation.

In the final league, published in July 2026, gpt-5.6-sol placed first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26. Firmulate’s stated rule is that partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”

All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding was strikingly simple: “Same diagnosis, same pitch — no signature.” Recognizing the right move and carrying it through proved to be different tests.

The clue was buried in the company’s files

The decisive competitor weakness was two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. The result points to a familiar challenge in education and other document-heavy fields: relevant evidence may be available, but an AI has to find and use it before a decision is made.

The test also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 explained its decision on the record: “Treat the request as a suspected approval-bypass / possible impersonation.” That restraint matters wherever staff handle sensitive student, family or institutional information.

Thorough work still needs discipline

Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, yet it finished last. The close was left on the table, and discipline slipped: it made write attempts into a locked department instead of escalating. A weaker version of the same weakness appeared in all four participants.

There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions, giving readers a way to test their own intuitions about how different systems behave.

The live company has 13 synthetic employees and real money mechanics: burn of €105k/month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules and versioned workdays. Readers can watch it at firmulate.com. Firmulate describes the experiment as watchable and ongoing; its published site excerpt reports that the live lab is running and that benchmark runs are queued.

For education leaders, the broader point is not that a synthetic software company predicts exactly how an AI would behave in a school. It is that agents should be evaluated on their decisions under pressure, including whether they follow through, protect trust and escalate when they lack authority. A controlled wargame can make those questions visible before an organization gives an AI access to consequential work.

From watching to a pilot

Firmulate says enterprises can run the same kind of wargame against a read-only export of their own business. The exercise tests crisis scenarios against that organization’s data and produces a board report with model rankings and weaknesses in its playbooks. The export is read-only; nothing writes back to real systems.

Put the decision-making to the test

For schools, education companies and other institutions exploring AI agents, the central question is whether a model can act reliably within real rules—not simply describe a good response. Firmulate’s experiment shows why follow-through, document reading and escalation belong in that evaluation.

To discuss a pilot using a read-only export of your organization’s business, visit firmulate.com/pilot.html or contact contact@firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI