Together AI raises $800M to scale open-source AI cloud for builders

Together AI, the operator of a cloud platform optimized for open-source AI models, has raised $800 million in a Series C round led by Aramco Ventures at an $8.3 billion valuation. For AI builders, this signals that open-source model inference and training infrastructure is attracting serious capital to compete with closed-model providers.

What happened

Together AI closed an $800 million Series C led by Aramco Ventures, with participation from Nvidia, Vista Equity Partners, General Catalyst, and others. The company is now valued at $8.3 billion. It disclosed that its annualized bookings topped $1.15 billion in Q2 and that it serves thousands of organizations, including LG AI research labs, Cohere, and the Mozilla Foundation.

The platform offers a serverless inference service that lets developers run open-source models without configuring GPUs or networking. Together AI claims its serverless environments provide about twice the performance of the fastest alternative. Beyond serverless, the company sells two dedicated infrastructure tiers for higher reliability and customization, plus a Batch Inference service that offers up to a 50% price reduction for non-real-time workloads.

Under the hood, the platform runs on Nvidia chips and a custom software engine called ATLAS. ATLAS uses speculative decoding, pairing a lightweight draft model with the main model to generate responses faster. Together AI claims this technique can speed up some inference workloads by up to 400%. The company also provides training clusters with thousands of GPUs, manageable via Kubernetes or Slurm, with automated detection and remediation of hardware issues.

Together AI plans to use the new capital to expand its public cloud capacity by 50x over the next five years and to enhance its training and inference features.

Why AI builders should care

For teams building AI products on open-source models, Together AI's funding and growth validate that open-source compute can be a viable alternative to closed APIs. The platform's serverless and batch inference options give builders flexibility to trade off latency for cost. The claimed 2x performance improvement over alternatives and up to 400% speedup via speculative decoding could reduce inference costs and latency for production workloads.

The training cluster features, including Kubernetes/Slurm management and automatic issue remediation, address real pain points for teams fine-tuning or training models at scale. Hardware failures during long training runs are a common source of wasted compute and time.

Practical implications

If you are evaluating Together AI for your stack, consider the following:

• Serverless inference is best for variable workloads where you don't want to manage infrastructure. The claimed performance advantage could lower your per-token cost.
• Batch Inference is useful for offline processing, data labeling, or any workload where responses don't need to be instant. The 50% discount makes it attractive for high-volume, non-latency-sensitive tasks.
• Training clusters with auto-remediation reduce the operational burden of managing GPU fleets. If you are fine-tuning models regularly, this could save engineering time.
• The platform is built on Nvidia hardware, so if you are already using CUDA-optimized models, migration should be straightforward.

Caveats

Several claims in this announcement are company-provided and not independently verified. The 2x performance claim and 400% speculative decoding speedup are based on Together AI's own testing. Actual results will vary by model, workload, and configuration. The Batch Inference discount of up to 50% likely depends on model size and queue priority.

While Together AI lists notable customers like Cohere and Mozilla, the depth of their usage is not detailed. The 50x capacity expansion plan is an ambitious target over five years and depends on continued capital availability and hardware supply.

Finally, the platform is optimized for open-source models. If your workflow depends on proprietary models (e.g., GPT-4, Claude), Together AI may not be a direct replacement.

FAQs

What is Together AI and what does it do?

Together AI operates an open-source AI-focused cloud platform that provides serverless inference, dedicated inference, and batch inference services for running open-source models. The platform is built on Nvidia GPUs and the custom ATLAS engine, which uses speculative decoding to accelerate workloads. It also offers training clusters with thousands of GPUs managed via Kubernetes or Slurm.

Who led Together AI's $800 million Series C?

The Series C round was led by Aramco Ventures, with participation from Nvidia, Vista Equity Partners, General Catalyst, and other institutional investors. The funding values Together AI at $8.3 billion.

What is the ATLAS engine and how does it relate to Together AI?

ATLAS is Together AI's custom software engine that powers its inference platform. It uses speculative decoding, where a lightweight draft model generates a quick response that the main model then verifies and refines. Together AI claims ATLAS can adapt the draft model to changing user requirements and speed up some inference workloads by up to 400%.

What customers use Together AI's platform and for what purposes?

Together AI reports that its platform is used by thousands of organizations, including LG AI research labs, Cohere, and the Mozilla Foundation. These customers use the platform to run open-source AI models at scale for inference and training workloads.