> For the complete documentation index, see [llms.txt](https://docs.visdom.virtuslab.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.visdom.virtuslab.com/visdom-ai-gateway/available-today.md). # Available today What works now, and how we know. Each item says where it comes from. **Tested** means the smoke test in the `visdom-ai-gateway` repository exercises it on every run. **Upstream** means the gateway product does it and the reference deployment has it switched on, but our test does not yet exercise it. **Visdom** means a Visdom product does it today, outside the gateway. ## LLM Gateway | Capability | Source | Notes | | ------------------------------------------------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | One OpenAI-compatible endpoint (`/v1/chat/completions`) | Tested | Clients use a model alias; the provider behind it is configuration | | Anthropic Messages endpoint (`/v1/messages`) on the same key | Tested | Native for Claude models; translated for others | | Virtual keys issued per team | Tested | A key belongs to a team; the provider credential never leaves the gateway | | Team budget with a soft alert and a hard limit | Tested | For example an alert at 8 of 10 USD (80%), a stop at 10 | | Hard stop when a key is over its budget | Tested | HTTP 429 | | Model allow-list per team | Tested | A model outside the list is refused with HTTP 403 | | Spend attributed to the team | Tested | Recorded per call from the price list, readable through the API | | Provider connectors | Upstream | Azure OpenAI, OpenAI, Anthropic (direct, Bedrock, Vertex AI), Vertex AI and Gemini, and any OpenAI-compatible server inside the network (vLLM, TGI, Ollama for development) | | Routing, retries and fallback between deployments | Upstream | Simple shuffle by default; lowest-cost, least-busy and latency strategies available | | Requests and tokens per minute per key and team | Upstream | | | Budget alerts to a webhook | Upstream | Soft budget reached and limit exceeded; configured in `config.yaml` | | Prompt and completion content kept out of the spend log | Configured | `turn_off_message_logging: true` in the reference configuration | | Admin UI | Upstream | At `/ui`, signed in with the master key | ## MCP Gateway | Capability | Source | Notes | | ---------------------------------------------------------------- | ---------- | --------------------------------------------------------------------------------------- | | Anonymous calls refused | Tested | HTTP 401 without a token | | Tool server registration and tool discovery | Tested | A server reachable over Streamable HTTP is registered; its tools appear in the registry | | Virtual servers: a curated set of tools under one endpoint | Tested | `/servers//mcp` | | Tool calls over MCP Streamable HTTP through the gateway | Tested | `initialize` and `tools/call`, protocol 2025-11-25 | | Admin UI | Upstream | At `/admin` | | Federation of many MCP servers behind one address | Upstream | | | REST and gRPC services exposed as MCP tools | Upstream | | | Per-server credentials stored encrypted | Upstream | The user never holds the tool's credential | | Protection against calls to unexpected internal addresses | Configured | SSRF protection on; only the networks in `MCP_ALLOWED_NETWORKS` are reachable | | MCP specification versions 2024-11-05 to 2025-11-25 | Upstream | See [API compatibility](/visdom-ai-gateway/api-compatibility.md) | | Plugins: deny lists, regex filters, PII filter, rate limits, OPA | Upstream | Not yet configured in the reference deployment | ## Around the gateway, in other Visdom products | Capability | Source | Notes | | ----------------------------------------------------------------- | --------------------- | ----------------------------------------------------------------------------- | | LLM proxy with a per-call cost and token ledger | Visdom (AI Tracing) | Anthropic Messages and OpenAI Responses; research preview | | Per-user upstream keys, stored encrypted, never shown to the user | Visdom (AI Tracing) | Plus per-user model routing and model-name rewrite | | Secret blocking before a prompt leaves | Visdom (AI Tracing) | Registered secret fingerprints; HTTP 403 before forwarding; off by default | | Signed, hash-chained evidence verifiable offline | Visdom (AI Tracing) | SHA-256 chain with Ed25519 signatures | | Per-agent MCP server and method allow-lists, human approvals | Visdom (Orchestrator) | For agents running as Orchestrator flows | | Workstation tools can reach only the gateway | Visdom (Sandcat) | Transparent proxy with an egress allow-list; secrets substituted in the proxy | ## Operations * **One compose file** for a single host; PostgreSQL for both gateways' state. * **Configured not to call home.** LiteLLM telemetry off; its price list read from the image rather than fetched at start-up; no licence activation. * **Mirrorable.** Three images, overridable per variable, for an internal registry. What the gateway does not do yet is on [Known limitations](/visdom-ai-gateway/known-limitations.md). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.visdom.virtuslab.com/visdom-ai-gateway/available-today.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.