Building AI applications in .NET has never been easier…
With Microsoft.Extensions.AI, you can create production-ready AI services without drowning in boilerplate code or vendor-specific implementations.
Today, I’ll show you how to build your first AI .NET application in under 30 minutes, and yes, running entirely on your local machine, for free, with no API keys required.
Why Microsoft.Extensions.AI is a good to start?
Before we dive into code, let’s address the elephant in the room.
Microsoft.Extensions.AI provides a unified abstraction layer over different AI providers, following the same patterns you already know from ASP.NET Core. Think of it as the ILogger or IHttpClientFactory for AI services, following the DIP (SOLID).
This way you can easily switch LLMs, add caching and not re-invert the wheel.
“But What About Semantic Kernel?”
Yes, Semantic Kernel exists, and it’s powerful, especially for complex orchestration scenarios, agents, and plugin systems. I use it regularly and will cover it in depth in my next post.
For today’s goal, building your first AI app in 30 minutes, Microsoft.Extensions.AI is the perfect choice. It’s the thin, elegant abstraction that gets you running immediately. Once you need more sophistication, you can layer in Semantic Kernel
The best part? If you’ve built anything with ASP.NET Core in the last 5 years, you already know how to use it.
Step 1: Install Ollama (3 minutes)
Ollama is a “Docker for LLMs”, it manages and runs open-source models locally.
You can download it here
The, download your first model, here are some examples:
ollama pull llama3.2
ollama pull mistral # 4GB - great for structured data
ollama pull phi3 # 2.2GB - Microsoft's compact model
Ollama runs a api on http://localhost:11434 by default. That’s all the setup needed!
Inside Microsoft.Extensions.AI, you’ll find Microsoft.Extensions.AI.Abstractions, the core package that defines the universal interfaces like IChatClient and IEmbeddingGenerator. This abstraction layer is what makes the magic happen: OpenAI, Gemini, Ollama, Azure OpenAI, and other providers all implement these same interfaces. That’s why you can switch between providers with just a configuration change, they all speak the same “language” defined by these abstractions.
Add the required packages:
dotnet add package Microsoft.Extensions.AI
dotnet add package OllamaSharp
dotnet add package Microfost.Extensions.AI.Abstractions
#Production
dotnet add package Microsoft.Extensions.AI.OpenAi
How to Use It
Once you’ve configured Microsoft.Extensions.AI with your chosen provider, using it is remarkably straightforward.
First, configure your AI client in program.cs:
builder.Services.AddCarter();
const string openAiModel = "gpt-4o-mini";
const string openAiKey = "my-key";
var ollamaUri = new Uri("http://localhost:11434");
const string ollamaModel = "phi3:latest";
var client = builder.Environment.IsDevelopment() ?
new OllamaApiClient(ollamaUri, ollamaModel) :
new ChatClient(openAiModel, openAiKey).AsIChatClient();
builder.Services.AddSingleton<IChatClient>(client);
You can get your client information from your appsettings, secrets or other environment, to simplify, we will use constants here.
This configuration elegantly switches between local Ollama in development (free, private, no API limits) and OpenAI in production (better quality, global scale). The ternary operator makes the intent crystal clear: develop locally, deploy to cloud.
Now, here’s the complete endpoint implementation:
public class QuestionsEndpoints : CarterModule
{
public QuestionsEndpoints() : base("/questions")
{
this.WithTags("Questions");
this.WithSummary("Handles AI question processing operations");
}
public override void AddRoutes(IEndpointRouteBuilder app)
{
app.MapPost("/", async (QuestionRequest request, IChatClient chatClient, CancellationToken ct) =>
{
var result = await chatClient.GetResponseAsync([
new(ChatRole.System, "You are a precise technical assistant. Provide direct, accurate answers limited to 100 words maximum. Focus on the most important information. Skip pleasantries and filler text, deliver only essential content."),
new(ChatRole.User, request.Prompt)
], cancellationToken: ct);
return Results.Ok(new QuestionResponse(result.Text));
})
.WithName("ProcessQuestion")
.WithSummary("Process a question using AI")
.WithDescription("Sends a question prompt to the AI service and returns the response")
.Produces<QuestionResponse>(200)
.Produces(400);
}
}
The beauty lies in what you’re not seeing here. Notice how IChatClient is injected directly into the endpoint, Carter and dependency injection handle this automatically. The AI provider switches based on environment without touching endpoint code.
The system prompt enforces concise responses (100 words max), perfect for quick, focused answers. This constraint actually improves response quality by forcing the AI to prioritize essential information.
Message array – The conversation context sent to the AI:
- ChatRole.System – Sets the AI’s behavior and constraints. Here we limit responses to 100 words and demand precision
- ChatRole.User – The actual user question from request.Prompt
Request:
{
"prompt": "What is Microsoft.Extensions.AI?"
}
Response:
{
"answer": "Microsoft.Extensions.AI (Msft AI) was an initiative by Microsoft to enable developers to build
intelligent applications with ease using tools for understanding language and context as well
as making predictions on data within apps, without requiring extensive machine learning expertise.
It integrated common APIs from Bing's Cognitive Services into .NET core libraries, allowing
seamless AI integration in desktop, mobile, server-based software. The project aimed at
democratizing access to AI capabilities for developers across various industries and was a part
of Microsoft's broader strategy on Artificial Intelligence innovation including Azure services
like Cognitive Services which offered these APIs as cloud solutions before the Msft AI initiative
became less prominent."
}
Adding Intelligent Caching: Stop Paying for Repeated Questions
Here’s a reality check: in production, users ask the same questions repeatedly.
Without caching, you’re paying for (or computing) the same AI responses over and over. That’s like recompiling your code for every request, wasteful and expensive.
Why Caching AI Responses is Critical
Cost Reduction: You’re not burning $ on duplicate responses.
Performance: Even local Ollama takes 1-3 seconds per response. Cache hits return in <10ms, that’s 100-300x faster. Your users feel the difference.
Scalability: AI providers have rate limits. OpenAI’s tier 1 allows 500 requests/minute. Caching prevents hitting these limits during traffic spikes.
Reliability: When AI providers have outages (yes, it happens), cached responses keep your app running for common queries.
Update your Program.cs configuration:
var client = builder.Environment.IsDevelopment() ?
new OllamaApiClient(ollamaUri, ollamaModel) :
new ChatClient(openAiModel, openAiKey).AsIChatClient();
if (builder.Environment.IsDevelopment())
{
builder.Services.AddDistributedMemoryCache();
}
else
{
builder.Services.AddStackExchangeRedisCache(options =>
{
options.Configuration = builder.Configuration.GetConnectionString("Redis");
});
}
var cache = builder.Services.BuildServiceProvider().GetRequiredService<IDistributedCache>();
var cachedClient = new ChatClientBuilder(client)
.UseDistributedCache(cache)
.Build();
builder.Services.AddSingleton<IChatClient>(cachedClient);
The endpoint code remains exactly the same:
public override void AddRoutes(IEndpointRouteBuilder app)
{
app.MapPost("/", async (QuestionRequest request, IChatClient cachedClient, CancellationToken ct) =>
{
var result = await cachedClient.GetResponseAsync([
new(ChatRole.System, "You are a precise technical assistant. Provide direct, accurate answers limited to 100 words maximum. Focus on the most important information. Skip pleasantries and filler text, deliver only essential content."),
new(ChatRole.User, request.Prompt)
], cancellationToken: ct);
return Results.Ok(new QuestionResponse(result.Text));
})
.WithName("ProcessQuestion")
.WithSummary("Process a question using AI")
.WithDescription("Sends a question prompt to the AI service and returns the response")
.Produces<QuestionResponse>(200)
.Produces(400);
}
Notice something beautiful?
The endpoint didn’t change at all. The parameter is still just IChatClient, whether it’s cached or not is a configuration concern, not a code concern.
You can also configure expiration time with UseDistributedCache and other details if you prefer.
Conclusion
We’ve just built a production-ready AI application in .NET using Microsoft.Extensions.AI, and it took us less than 30 minutes from zero.
We wrote 15 lines of endpoint code. That’s it. No wrappers, no abstractions, no ceremony. Just a clean Carter module that accepts a question and returns an answer.
The key insight? Treat AI like any other infrastructure service – abstract it, inject it, swap it. Today Ollama, tomorrow OpenAI, next year whatever comes next. Your code doesn’t care.
With caching, we turned unpredictable AI costs into fixed expenses. Response times dropped from 1.2 seconds to 15ms. That’s not optimization, that’s transformation.
This isn’t a proof of concept. This is production code that scales from your laptop to the cloud without changes.
Now stop reading and start shipping. The abstractions are solid. The patterns are proven. Your AI-powered application awaits.
Thank you for reading.
See you next time!
