Connecting to most available LLM Providers
This post summarizes the video "How we connect different model providers".
It provides the necessary NuGet packages, setup steps, and C# code for integrating various LLMs into the Microsoft Agent Framework.
Core Architecture
The framework is designed around the IChatClient interface (from Microsoft.Extensions.AI). Any provider that can be wrapped in this interface can power a Microsoft Agent.
1. OpenAI
var client = new OpenAIClient("YOUR_API_KEY");
IChatClient chatClient = client.AsChatClient("gpt-4o");
var agent = new ChatClientAgent(chatClient);
await agent.RunAsync("Hello!");
2. Google Gemini
var chatClient = new GoogleAiChatClient(apiKey: "YOUR_KEY", modelId: "gemini-1.5-pro");
var agent = new ChatClientAgent(chatClient);
3. Anthropic
Source: console.anthropic.com
NuGet: Community Anthropic SDK + Microsoft.Extensions.AI.
Note: MaxTokens is mandatory in ChatOptions.
var client = new AnthropicClient("YOUR_KEY");
var chatClient = client.AsChatClient();
var agent = new ChatClientAgent(chatClient, new ChatOptions {
ModelId = "claude-3-5-sonnet",
MaxTokens = 1000
});
4. Mistral
var client = new MistralClient("YOUR_KEY");
var chatClient = client.AsChatClient();
var agent = new ChatClientAgent(chatClient, new ChatOptions { ModelId = "mistral-large-latest" });
5. XAI (Grok)
var options = new OpenAIClientOptions { Endpoint = new Uri("[https://api.x.ai/v1](https://api.x.ai/v1)") };
var client = new OpenAIClient("YOUR_KEY", options);
IChatClient chatClient = client.AsChatClient("grok-beta");
6. Azure OpenAI
var client = new AzureOpenAIClient(new Uri("YOUR_ENDPOINT"), new AzureKeyCredential("YOUR_KEY"));
IChatClient chatClient = client.AsChatClient("YOUR_DEPLOYMENT_NAME");
7. Microsoft Foundry (Cloud)
Source: Azure AI Foundry.
NuGet: Microsoft.Agents.AI.Azure.AI
Note: Uses Azure Credentials (CLI/Managed Identity) instead of API keys.
var projectClient = new AIProjectClient("YOUR_CONNECTION_STRING", new DefaultAzureCredential());
var agent = await projectClient.GetAgentAsync("AGENT_ID");
// Turn Foundry Agent into Framework Agent
var frameworkAgent = agent.AsAgent();
8. Amazon Bedrock
// Set AWS_BEARER_TOKEN_BEDROCK environment variable
var runtime = new AmazonBedrockRuntimeClient();
IChatClient chatClient = runtime.AsChatClient("anthropic.claude-3-sonnet");
9. Ollama (Local)
var uri = new Uri("http://localhost:11434");
var client = new OllamaApiClient(uri);
IChatClient chatClient = client.AsChatClient("llama3");
10. Microsoft Foundry Local
Source: WinGet install Microsoft Foundry Local.
NuGet: Microsoft.AI.FoundryLocal
Note: Runs models locally using system NPU/GPU/CPU.
var model = await FoundryLocal.StartModelAsync("phi3");
var client = new OpenAIClient(new OpenAIClientOptions { Endpoint = model.Endpoint });
IChatClient chatClient = client.AsChatClient(model.ModelId);
11-14. Multi-Model Portals (OpenRouter, Together.ai, Cohere, HuggingFace)
These providers use the OpenAI-compatible standard. Use Microsoft.Agents.AI.OpenAI and change the endpoint.
Provider | Endpoint |
|---|
OpenRouter | https://openrouter.ai/api/v1
|
Together.ai | https://api.together.xyz/v1
|
Cohere | https://api.cohere.ai/compatibility/v1
|
HuggingFace | https://router.huggingface.co/v1
|
Generic Code Pattern:
var options = new OpenAIClientOptions { Endpoint = new Uri("PROVIDER_ENDPOINT") };
var client = new OpenAIClient("YOUR_API_KEY", options);
IChatClient chatClient = client.AsChatClient("MODEL_NAME");
15. GitHub Models
var client = new ChatCompletionsClient(new Uri("[https://models.inference.ai.azure.com]
(https://models.inference.ai.azure.com)"), new AzureKeyCredential("GITHUB_PAT"));
IChatClient chatClient = client.AsChatClient();
var agent = new ChatClientAgent(chatClient, new ChatOptions { ModelId = "gpt-4o" });
Key Takeaway: The "magic" is the IChatClient abstraction. Once you have an IChatClient, you can pass it to any Agent in the framework, regardless of the underlying LLM provider.
