> For the complete documentation index, see [llms.txt](https://docs.visdom.virtuslab.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.visdom.virtuslab.com/visdom-ai-gateway/user-guide/models-and-routing.md).

# Models and routing

The model catalogue – what clients can ask for, and what serves it.

Clients never name a provider. They ask for an **alias**, and `deploy/compose/litellm/config.yaml` decides what serves it. Switching a provider, a region or a model version is a change to that file, reviewed like any other, and no client notices.

## An alias

```yaml
model_list:
  - model_name: claude-sonnet            # what clients ask for
    litellm_params:
      model: anthropic/claude-sonnet-5   # what serves it
      api_key: os.environ/ANTHROPIC_API_KEY
```

Credentials are read from the environment (`os.environ/…`), never written in the file.

## Several deployments behind one alias

Repeat the alias with a different deployment and the gateway spreads calls across them and fails over when one is down:

```yaml
  - model_name: gpt-general
    litellm_params:
      model: azure/gpt-5-eu
      api_base: os.environ/AZURE_API_BASE_EU
      api_key: os.environ/AZURE_API_KEY_EU
  - model_name: gpt-general
    litellm_params:
      model: azure/gpt-5-eu2
      api_base: os.environ/AZURE_API_BASE_EU2
      api_key: os.environ/AZURE_API_KEY_EU2
```

This is also how one alias can use two subscriptions, for example one per department.

## Fallback to another model

```yaml
router_settings:
  num_retries: 2
  fallbacks: [{"claude-sonnet": ["gpt-general"]}]
```

A fallback is one client request and more than one provider call; each provider call is costed at what it actually cost.

## Models inside the network

Any server with an OpenAI-compatible API – vLLM, TGI, or Ollama for development – is one more entry:

```yaml
  - model_name: local-coder
    litellm_params:
      model: openai/qwen3-coder
      api_base: http://vllm.internal:8000/v1
      api_key: not-used
    model_info:
      input_cost_per_token: 0.0000002    # the organisation's own GPU rate, for chargeback
      output_cost_per_token: 0.0000006
```

## Pinning and migrating a version

Point the alias at a dated model version rather than a moving name. Moving every client to a new version is a one-line change to the alias; moving some clients first is a second alias and a change to their teams' allow-lists.

## Parameters a provider does not support

The reference configuration sets `drop_params: true`: a parameter the target provider does not accept is removed instead of failing the call. Turn it off where silent differences between providers matter more than availability.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.visdom.virtuslab.com/visdom-ai-gateway/user-guide/models-and-routing.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.