Three AI workloads, the problems we found, and the changes that made them work better.
We wanted a lower-cost model to handle everyday CRM work: updating contacts, creating opportunities, and qualifying leads. The test was whether it could leave the right changes in the CRM.
Read the story75.5%
lower model cost than Sonnet 4.6
A small model could handle scheduling and incident tasks, but spent too long generating its answers. We investigated what it was writing and found a way to make it much faster.
Read the story5.2×
faster median model response than Sonnet 4.6
The small model passed every task check on 92.2% of attempts; Sonnet passed 100%. Both ran 30 AutomationBench workflows, with 90 attempts per model.
A film can attract thousands of enthusiastic comments. We tested an open model’s ability to find people planning to see it in theaters, with full-table model cost of $2.82.
Read the story$2.82
model cost for 39,962 comments