Last summer, in early August, we had our half-year business review with Amazon. It was a proud moment for us. It was 8 months after we wrote the first line of code for Gigi, and would be the first time that we had the opportunity to demo the new product to Amazon's ad leadership. Like most early AI products, Gigi's initial use cases for demonstrating value were limited and we constructed a highly curated demo experience for the review.

We were able to get to the "aha" moment pretty quickly. Canned prompts and demo-ware campaign builds opened the eyes of Amazon's ad leadership. An Amazon leader was sufficiently intrigued and wanted to test Gigi's limits. In what now feels like ages ago, last summer LLMs struggled with open-ended prompts. Ask the LLM a very specific question, it could nail an answer. But the Amazon leader asked a classic open-ended media question: "I have $50k in extra budget next month, how should I allocate it and why?" Gigi tried. But Gigi errored out. The LLMs weren't good enough.

Two days later, GPT-5 was released. Naturally, we wanted to measure how much of a step change this new model was and we asked that same question: how would you allocate an incremental $50k next month and why? One of our engineers ran the prompt locally on his computer. We saw the chain of thought. Gigi was using new tool calls. It took Gigi 10 minutes, but Gigi built a beautiful data-driven media plan with a perfectly rational explanation on how the budget should be allocated. We felt like we were living in the future. Last summer, which frontier model you picked determined whether you won. It doesn't anymore.

In December last year, the step-change improvements in frontier model sophistication fundamentally altered the efficacy of coding agents, and thereby the entire software industry. Over the past month, a step-change improvement in open source models and model providers outside of OpenAI and Anthropic has altered the economics and product experiences of vertical AI agents like Gigi. SOTA frontier models are no longer demonstrably better than off-the-shelf open-source or cheaper models like Meta's Muse Spark. SOTA frontier models have become overqualified. It's economically irrational to run a model like Anthropic's Fable for >95% of use cases. Model selection and model routing are now critical for every vertical AI agent.

Here's how we think about model selection at Gigi: While we're always cognizant of managing compute costs with target gross margins, we strongly believe that cost alone should never be the primary driver of model decisions. Our decision to use (or not use) a model will be based on the following factors, all tied to the customer experience.

• Performance - can the model meet our internal benchmarks and provide an ideal outcome for our customers?

• Speed - can the model provide that ideal response as fast as possible for our customer?

• Cost - what model is able to meet our speed and performance requirements at the cheapest cost?

• Customizability - how can we fine-tune and post-train an open source model to better meet our customers' needs?

The $50k question that broke Gigi last August is now table stakes. Every model we evaluate can answer it, in seconds, for a fraction of what it cost us a year ago. As intelligence becomes abundant and models become more commoditized, we're particularly excited about the new set of use cases that Gigi can offer our customers. As an example, Gigi can now run long-running autonomous tasks in the background without cost being a concern.

The simple days of plugging a SOTA frontier model into a vertical AI agent are done. For companies like Gigi, the tooling we build around the models (aka "the harness") becomes as important as the choice of underlying model. That's good news. A year ago we were rate-limited by the models. Now we're rate-limited by our own imagination about where to point them.