LLM Providers - MCPJam Inspector

The SDK supports 10 built-in LLM providers and allows custom provider configuration for any OpenAI or Anthropic-compatible endpoint. It can also run models on MCPJam’s own credits, so an eval in CI needs no provider key at all.

Model String Format

All models use the format provider/model-id:

"anthropic/claude-sonnet-4-20250514"
"openai/gpt-4o"
"google/gemini-1.5-pro"

MCPJam-hosted (no provider key)

Prefix a model with mcpjam/ and MCPJam calls the provider for you, billing your organization’s credits. The only secret you need is an MCPJam API key — which is usually already in your CI for saving results.

Property	Value
Prefix	`mcpjam/`
Environment Variable	`MCPJAM_API_KEY` (an `sk_…` key from Settings → API keys)

const agent = new HostRunner({
  tools,
  // The vendor model id follows the prefix.
  model: "mcpjam/anthropic/claude-sonnet-4.5",
  apiKey: process.env.MCPJAM_API_KEY!,
});

apiKey falls back to MCPJAM_API_KEY, so in CI you can leave it out entirely.

Available models

mcpjam/anthropic/claude-* and mcpjam/openai/gpt-5*. Other vendors are not hosted — use their own prefix and key.

Which project pays

By default the API key’s organization Default project, which is also where results are saved — so nothing extra to configure. Set MCPJAM_PROJECT_ID to bill a specific project.

Cost

Spend appears under the organization’s usage. Eval runs also show an estimated cost per iteration, priced from the tokens used at MCPJam’s rate for the model. It can differ slightly from the credits billed.

Limits

Handled for you, but worth knowing about if you see one in a log:

Leases last 30 minutes and are renewed automatically.
Each lease has its own spend and call ceiling; the SDK mints a fresh one when it is used up.
Your organization’s credit balance and spend budget are checked on every call. “Spending limit reached” means add credits — it is not retried.
A suite revokes its leases when it finishes. If you build runs by hand, call releaseMcpjamModelLeases() in an afterAll.

import { releaseMcpjamModelLeases } from "@mcpjam/sdk";

afterAll(async () => {
  await releaseMcpjamModelLeases();
});

Built-in Providers

Anthropic

Property	Value
Prefix	`anthropic/`
Environment Variable	`ANTHROPIC_API_KEY`

Models

Model ID	Description
`claude-sonnet-4-20250514`	Latest Sonnet (recommended)
`claude-3-5-sonnet-20241022`	Claude 3.5 Sonnet
`claude-3-haiku-20240307`	Fast, efficient
`claude-3-opus-20240229`	Most capable

Example

const agent = new HostRunner({
  tools,
  model: "anthropic/claude-sonnet-4-20250514",
  apiKey: process.env.ANTHROPIC_API_KEY,
});

OpenAI

Property	Value
Prefix	`openai/`
Environment Variable	`OPENAI_API_KEY`

Models

Model ID	Description
`gpt-4o`	Latest GPT-4 Omni (recommended)
`gpt-4o-mini`	Fast, cost-effective
`gpt-4-turbo`	GPT-4 Turbo
`gpt-4`	Original GPT-4
`gpt-3.5-turbo`	Fast, economical

Example

const agent = new HostRunner({
  tools,
  model: "openai/gpt-4o",
  apiKey: process.env.OPENAI_API_KEY,
});

Azure OpenAI

Property	Value
Prefix	`azure/`
Environment Variables	`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`

Environment Variables

Variable	Required	Description
`AZURE_OPENAI_API_KEY`	Yes	API key
`AZURE_OPENAI_ENDPOINT`	Yes	Endpoint URL
`AZURE_OPENAI_API_VERSION`	No	API version

Example

// Set environment variables first:
// AZURE_OPENAI_API_KEY=...
// AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com

const agent = new HostRunner({
  tools,
  model: "azure/gpt-4o",
  apiKey: process.env.AZURE_OPENAI_API_KEY,
});

Amazon Bedrock

Property	Value
Prefix	`bedrock/`
Environment Variables	`AWS_BEARER_TOKEN_BEDROCK`, `AWS_REGION`

Authentication uses a Bedrock API key (bearer token). The AWS region is read from AWS_REGION unless a full runtime endpoint is supplied via baseUrls.bedrock (e.g. https://bedrock-runtime.us-east-1.amazonaws.com).

Models

Use any Bedrock model or inference profile ID your AWS account has been granted access to:

Model ID	Description
`us.anthropic.claude-sonnet-4-5-20250929-v1:0`	Claude Sonnet 4.5 (US inference profile)
`anthropic.claude-3-5-haiku-20241022-v1:0`	Claude 3.5 Haiku
`us.amazon.nova-pro-v1:0`	Amazon Nova Pro
`meta.llama3-70b-instruct-v1:0`	Meta Llama 3 70B

Environment Variables

Variable	Required	Description
`AWS_BEARER_TOKEN_BEDROCK`	Yes	Bedrock API key (bearer token)
`AWS_REGION`	Yes	AWS region (e.g. `us-east-1`), unless `baseUrls.bedrock` is set

Example

// Set environment variables first:
// AWS_BEARER_TOKEN_BEDROCK=...
// AWS_REGION=us-east-1

const agent = new HostRunner({
  tools,
  model: "bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0",
  apiKey: process.env.AWS_BEARER_TOKEN_BEDROCK,
});

Google (Gemini)

Property	Value
Prefix	`google/`
Environment Variable	`GOOGLE_GENERATIVE_AI_API_KEY`

Models

Model ID	Description
`gemini-1.5-pro`	Most capable (recommended)
`gemini-1.5-flash`	Fast, efficient
`gemini-pro`	Original Gemini Pro

Example

const agent = new HostRunner({
  tools,
  model: "google/gemini-1.5-pro",
  apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});

Mistral

Property	Value
Prefix	`mistral/`
Environment Variable	`MISTRAL_API_KEY`

Models

Model ID	Description
`mistral-large-latest`	Most capable
`mistral-medium-latest`	Balanced
`mistral-small-latest`	Fast, economical
`open-mistral-7b`	Open source
`open-mixtral-8x7b`	Open source MoE

Example

const agent = new HostRunner({
  tools,
  model: "mistral/mistral-large-latest",
  apiKey: process.env.MISTRAL_API_KEY,
});

DeepSeek

Property	Value
Prefix	`deepseek/`
Environment Variable	`DEEPSEEK_API_KEY`

Models

Model ID	Description
`deepseek-chat`	General chat
`deepseek-coder`	Code-focused

Example

const agent = new HostRunner({
  tools,
  model: "deepseek/deepseek-chat",
  apiKey: process.env.DEEPSEEK_API_KEY,
});

Ollama (Local)

Property	Value
Prefix	`ollama/`
Default URL	`http://localhost:11434`

Models

Any locally installed model:

llama3
codellama
mistral
mixtral
etc.

Ensure Ollama is running locally before use. The API key parameter is required but not used, so pass any string.

Example

const agent = new HostRunner({
  tools,
  model: "ollama/llama3",
  apiKey: "ollama", // Required but not used
});

OpenRouter

Property	Value
Prefix	`openrouter/`
Environment Variable	`OPENROUTER_API_KEY`

Access many models through one API:

Model ID	Description
`anthropic/claude-3-opus`	Claude 3 Opus via OpenRouter
`openai/gpt-4-turbo`	GPT-4 Turbo via OpenRouter
`google/gemini-pro`	Gemini via OpenRouter
`meta-llama/llama-3-70b-instruct`	Llama 3 70B

Example

const agent = new HostRunner({
  tools,
  model: "openrouter/anthropic/claude-3-opus",
  apiKey: process.env.OPENROUTER_API_KEY,
});

xAI (Grok)

Property	Value
Prefix	`xai/`
Environment Variable	`XAI_API_KEY`

Models

Model ID	Description
`grok-beta`	Grok beta

Example

const agent = new HostRunner({
  tools,
  model: "xai/grok-beta",
  apiKey: process.env.XAI_API_KEY,
});

Custom Providers

Add your own OpenAI or Anthropic-compatible endpoints.

CustomProvider Type

interface CustomProvider {
  name: string;
  protocol: "openai-compatible" | "anthropic-compatible";
  baseUrl: string;
  modelIds: string[];
  useChatCompletions?: boolean;
  apiKeyEnvVar?: string;
}

Properties

Property	Type	Required	Description
`name`	`string`	Yes	Provider identifier (used in model string)
`protocol`	`string`	Yes	`"openai-compatible"` or `"anthropic-compatible"`
`baseUrl`	`string`	Yes	API endpoint URL
`modelIds`	`string[]`	Yes	Available model IDs
`useChatCompletions`	`boolean`	No	Use `/chat/completions` endpoint
`apiKeyEnvVar`	`string`	No	Custom env var name for API key

OpenAI-Compatible Example

const agent = new HostRunner({
  tools,
  model: "my-openai/gpt-4",
  apiKey: process.env.MY_API_KEY,
  customProviders: {
    "my-openai": {
      name: "my-openai",
      protocol: "openai-compatible",
      baseUrl: "https://api.my-provider.com/v1",
      modelIds: ["gpt-4", "gpt-3.5-turbo"],
    },
  },
});

Anthropic-Compatible Example

const agent = new HostRunner({
  tools,
  model: "my-anthropic/claude-3-sonnet",
  apiKey: process.env.MY_API_KEY,
  customProviders: {
    "my-anthropic": {
      name: "my-anthropic",
      protocol: "anthropic-compatible",
      baseUrl: "https://api.my-provider.com",
      modelIds: ["claude-3-sonnet", "claude-3-haiku"],
    },
  },
});

LiteLLM Example

const agent = new HostRunner({
  tools,
  model: "litellm/gpt-4",
  apiKey: process.env.LITELLM_API_KEY,
  customProviders: {
    litellm: {
      name: "litellm",
      protocol: "openai-compatible",
      baseUrl: "http://localhost:8000",
      modelIds: ["gpt-4", "claude-3-sonnet", "gemini-pro"],
      useChatCompletions: true, // Required for LiteLLM
    },
  },
});

Helper Functions

parseLLMString()

Parse a model string into provider and model ID.

import { parseLLMString } from "@mcpjam/sdk";

parseLLMString(modelString: string): { provider: string; modelId: string }

Example

const { provider, modelId } = parseLLMString("anthropic/claude-sonnet-4-20250514");
// provider: "anthropic"
// modelId: "claude-sonnet-4-20250514"

createModelFromString()

Create a provider model instance directly.

import { createModelFromString } from "@mcpjam/sdk";

createModelFromString(
  llmString: string,
  options: CreateModelOptions
): ProviderLanguageModel

Example

const model = createModelFromString(
  "openai/gpt-4o",
  { apiKey: process.env.OPENAI_API_KEY }
);

createCustomProvider()

Create a custom provider configuration object.

import { createCustomProvider } from "@mcpjam/sdk";

createCustomProvider(config: CustomProvider): CustomProvider

Example

const provider = createCustomProvider({
  name: "my-litellm",
  protocol: "openai-compatible",
  baseUrl: "http://localhost:8000",
  modelIds: ["gpt-4", "claude-3-sonnet"],
  useChatCompletions: true,
});

Environment Variables Summary

Provider	Variable
Anthropic	`ANTHROPIC_API_KEY`
OpenAI	`OPENAI_API_KEY`
Azure	`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`
Amazon Bedrock	`AWS_BEARER_TOKEN_BEDROCK`, `AWS_REGION`
Google	`GOOGLE_GENERATIVE_AI_API_KEY`
Mistral	`MISTRAL_API_KEY`
DeepSeek	`DEEPSEEK_API_KEY`
OpenRouter	`OPENROUTER_API_KEY`
xAI	`XAI_API_KEY`

Testing Across Providers - Conceptual guide
HostRunner Reference - Agent configuration

​Model String Format

​MCPJam-hosted (no provider key)

​Available models

​Which project pays

​Cost

​Limits

​Built-in Providers

​Anthropic

​Models

​Example

​OpenAI

​Models

​Example

​Azure OpenAI

​Environment Variables

​Example

​Amazon Bedrock

​Models

​Environment Variables

​Example

​Google (Gemini)

​Models

​Example

​Mistral

​Models

​Example

​DeepSeek

​Models

​Example

​Ollama (Local)

​Models

​Example

​OpenRouter

​Example

​xAI (Grok)

​Models

​Example

​Custom Providers

​CustomProvider Type

​Properties

​OpenAI-Compatible Example

​Anthropic-Compatible Example

​LiteLLM Example

​Helper Functions

​parseLLMString()

​Example

​createModelFromString()

​Example

​createCustomProvider()

​Example

​Environment Variables Summary

​Related

Model String Format

MCPJam-hosted (no provider key)

Available models

Which project pays

Cost

Limits

Built-in Providers

Anthropic

Models

Example

OpenAI

Models

Example

Azure OpenAI

Environment Variables

Example

Amazon Bedrock

Models

Environment Variables

Example

Google (Gemini)

Models

Example

Mistral

Models

Example

DeepSeek

Models

Example

Ollama (Local)

Models

Example

OpenRouter

Example

xAI (Grok)

Models

Example

Custom Providers

CustomProvider Type

Properties

OpenAI-Compatible Example

Anthropic-Compatible Example

LiteLLM Example

Helper Functions

parseLLMString()

Example

createModelFromString()

Example

createCustomProvider()

Example

Environment Variables Summary

Related