Model Vault Overview

Model Vault is a Cohere-managed inference environment for deploying and serving Cohere’s models and leading third-party open models, such as DeepSeek, GLM, MiMo, Gemma, Nemotron, GPT-OSS, Phi, and Qwen, on isolated, dedicated infrastructure. It gives you full control over model selection, scaling, and performance monitoring, without you operating the underlying serving stack. Zero Data Retention is on by default, so prompts and responses are not retained.

Because your infrastructure isn’t shared with other customers, you get the security and isolation of private hosting with the convenience of an API: no noisy neighbors, no rate limits, and predictable performance at scale.

You manage all of your vaults in one place from the Model Vault app, where you can track spend, usage hours, and activity across them. Model Vault comes in two types, Standard and Encrypted. The deployment and management experience is the same; the only difference is the level of data protection (see how they compare below).

vault.cohere.com

The Model Vault home page showing token spend, vault counts, and usage hours, a usage-over-time chart, and separate rows of Encrypted vault and Standard vault cards.

Why Model Vault

Dedicated infrastructure

Your load balancer, serving middleware, inference servers, and GPU accelerators are dedicated to you, so there are no noisy neighbors competing for capacity.

Fully managed

Create a vault from the Model Vault app and let Cohere handle maintenance, deployments, updates, and scaling. There’s no serving stack for you to operate.

Models and GPUs

Pick a Cohere model or a leading open model, then the GPU it runs on to match your latency, throughput, and cost needs.

Elastic, unthrottled capacity

Set a minimum and maximum replica, with autoscaling available. You get dedicated throughput with no rate limits, and pay an hourly rate for each GPU your replicas run on.

Built for production

Real-time monitoring of request rates, latency, token throughput, and GPU utilization helps you tune capacity for production workloads.

Standalone or with North

Call a vault directly over the API, or use it as the inference backend for North.

Sovereign inference

Model Vault lets you decide how your AI runs:

Where your data lives: run inference in a supported region (the United States, the European Union, or Canada) to help meet data residency and regulatory requirements.
Who can access it: every vault runs on isolated, dedicated infrastructure. With Model Vault Encrypted, no one but you can see your prompts or responses, including Cohere and the cloud provider.
Which models you use: choose the right model for each task from Cohere’s models and leading open models, and run all of them with the same protections. See Standard Vault and Encrypted Vault supported models.

How it works

Create a vault

In the Model Vault app, create a vault: name it, choose its region, add models and choose the GPU each runs on, and set each model’s replica range.

Get your endpoint

Once the vault is Ready, copy its endpoint URL and model name from the vault’s details page (see Managing Vaults).

Call it over the API

Point the Cohere SDK’s base_url at your vault endpoint and send chat, embed, or rerank requests, or use an OpenAI-compatible client for open models (see Calling a Vault over the API).

Two types of vault

Standard Vault

A Cohere-managed deployment on dedicated infrastructure with data protected in transit and at rest. Best when you want dedicated inference without managing the serving stack.

EXPLORE STANDARD VAULT →

Encrypted Vault

Everything in a Standard Vault, plus confidential computing: prompts, responses, and everything in between stay protected end to end inside hardware-backed trusted execution environments, with verifiable remote attestation. Best for regulated or highly sensitive workloads.

EXPLORE ENCRYPTED VAULT →

How they compare

	Standard Vault	Encrypted Vault
Managed deployment on dedicated infrastructure	Yes	Yes
Cohere models and open models	Yes	Yes
Zero Data Retention by default	Yes	Yes
Regions	United States, European Union, Canada	United States (European Union and Canada coming soon)
Home page, monitoring, usage & billing	Yes	Yes
Use standalone or with North	Yes	Yes
Data protected in transit and at rest	Yes	Yes
Data protected in use (confidential computing)	No	Yes
Verifiable remote attestation	No	Yes
Compliance support (GDPR, HIPAA, SOC 2)	Supported	Supported, plus verifiable attestation evidence
Supported models	Standard vault models	Encrypted vault models
Pricing	Per GPU, per hour (pricing page)	Per GPU, per hour (pricing page)

How this documentation is organized

Start with the shared sections that cover the day-to-day flow for every vault, regardless of type:

Deploy & manage: Home Page, Creating a Vault, Regions, and Managing Vaults.
Operate & observe: Monitoring.

Then dive into the section for your vault type for what’s specific to it, including how to call it over the API:

Standard Vault: supported models and calling the API (Cohere SDK, raw HTTP, or OpenAI-compatible).
Encrypted Vault: supported models, calling the API through the attestation-verifying Cohere OHTTP proxy and the confidential-computing deep dive (confidential computing, security model, remote attestation, key management and compliance).

And to connect a vault to North:

Model Vault with North: use a vault as the inference backend for North.

Get started

New to Model Vault? Start with the Quickstart.
Need confidential computing? See Encrypted Vaults.