Cedar is the AI native workspace to support account executives and sales teams across the entire mid-funnel, using LLMs in every aspect of the product. Through the Cedar platform, account executives and client-facing teams are supported by agents on every step of the sales journey, ensuring their pipelines stay healthy, opportunities close efficiently, and clients feel that reps are attentive to their needs.
The challenge.
Just 18 months after launching, Cedar’s user growth meant that traction and revenue were on the rise, but scaling in both user numbers and agents built meant that they would soon be spending upwards of $2.5M per month on inference; more than they could sustain as a seed startup. Cedar faced a choice: slow down growth and hiring in favor of saving money on their specifications—at the cost of both new users and existing user experience—or find a way to reduce their back-end inference costs.
The approach.
Orchestra stepped in with the solution. Orchestra’s model optimization and routing platform enabled Cedar’s engineers to run task-specific simulations for information retrieval, structured data extraction, summarization, and text generation to compare the performance of a frontier model against the performance of other available models on the same work.
With Orchestra’s task-specific evals in place for meeting summaries, CRM updates, and recommendations for next steps, Cedar’s engineers compared outputs from different models side by side, and assessed them against the level of quality expected by their users. Orchestra then routed each task to the most cost-effective model for the work. Orchestra’s simulations gave Cedar a way to use their own judgment and client feedback to evaluate lower-cost alternatives to frontier models. With qualitative benchmarks in place, Orchestra then facilitated Cedar’s training of proprietary models that meet the same level of quality at a fraction of the cost.
The results.
On the first day of partnering with Orchestra to set benchmarks, build evals, and route tasks to more efficient models, Cedar immediately reduced their inference costs for an extraction task by 71%. After their second round of optimization, Cedar further reduced their per-task inference costs by 83 to 97.5% on a range of tasks.
“At the beginning, we were just using Opus,” Cedar CEO and Founder Jesse Li said of the process. “We were basically margin negative. With Orchestra, as we got more and more data, being able to fine tune and develop the experience to run evals and move to open source has been insanely helpful.”
What's next.
“We've literally changed the fundamental economics of our business, and there's still so much we can do.”
Jesse Li, CEO and Founder at Cedar