Open-weight AI models gain traction as startups wrestle with OpenAI and Anthropic costs

Rising costs from OpenAI and Anthropic are pushing startups to re-evaluate their reliance on proprietary models and pivot toward cheaper open-weight AI models. The $15.6 billion legal-tech startup Harvey is the most prominent example: after a March AI agent update caused token usage to surge twentyfold, its gross margins dropped from about 50% to -50% by June, according to Bloomberg. Harvey responded by building its own model on Moonshot's Kimi K3, and margins turned positive again.

Harvey's margin collapse shows the cost of renting models

Harvey built its business around training models like OpenAI's GPT-4 for legal work. But after a March update to its AI agents, customer usage spiked and the per-token costs from OpenAI and Anthropic crushed its margins. Both companies now charge enterprises for model usage on top of base subscription fees, a shift that punished Harvey's "tokenmaxxing" usage pattern. The company stopped renting and in August released its first in-house model, post-trained on Kimi K3 from the Chinese lab Moonshot. Gross margins turned positive again after that launch and other changes, people familiar with the work told Bloomberg.

The broader shift toward model ownership

Harvey is not alone. Abridge is building a clinical model on Nvidia's open weights, Decagon now routes 80% of customer queries through its own models, and Ramp is weighing training for the first time. Sequoia Capital and General Catalyst are funding the shift. The pattern reflects a fundamental change in AI deployment strategy: as proprietary API costs eat into margins, startups are seeking cost predictability and data control through model ownership. Chinese open-weight models are reportedly 60 to 90% cheaper than leading Anthropic and OpenAI models for certain tasks.

What this means for AI builders

For teams building AI products, the economics of open-weight models change the calculus. Instead of paying per-token fees that scale with usage, you can invest in fixed training costs and run inference on your own infrastructure. That matters most for agent-heavy workflows where token usage can explode. However, owning a model is not always the right answer. Menlo Ventures partner Matt Kraning called it "a lot of cosplay" in most cases, and Anthropic noted that Harvey still needs Opus for its hardest tasks. The decision depends on your data volume, model quality requirements, and internal ML capabilities.

Caveats and open questions

The evidence comes primarily from Bloomberg and related coverage, so it reflects specific company experiences rather than universal practice. Not every startup will achieve positive margins after moving to open-weight models; outcomes depend on data costs, model quality, and internal capabilities. The shift also raises questions about supplier switching rights under the EU Data Act, though those cover customers leaving suppliers, not suppliers leaving customers. For now, the trend is clear: as API costs rise, more builders are treating model ownership as a viable option rather than a distraction.