Overview
This guide will walk you through creating a simple chat application in .NET that interacts with a locally hosted AI model. Using the Microsoft.Extensions.AI library, you can communicate with an AI model without relying on cloud services. This provides better privacy, reduced latency, and cost efficiency.
Prerequisites
- Install .NET 8.0 or a later version.
- Download and install Ollama for local AI execution.
- Install Visual Studio Code (optional).
Setting Up the Local AI Model
Follow these steps to configure and run an AI model on your machine.
Step 1: Verify Ollama Installation
Open a terminal and check if Ollama is installed by running:
ollamaIf installed, you’ll see a list of available commands.
Step 2: Start the Ollama Server
Start the Ollama service by running:
ollama serveStep 3: Download and Run the AI Model
Download the phi3:mini model from Ollama’s registry:
ollama pull phi3:miniOnce downloaded, start the model with:
ollama run phi3:mini
Creating the .NET Chat App
Step 1: Create a New .NET Console Application
Navigate to an empty directory and create a new console app:
dotnet new console -o LocalAI
Step 2: Install Required Package
Add the Microsoft.Extensions.AI.Ollama package to your project:
dotnet add package Microsoft.Extensions.AI.Ollama --prereleaseStep 3: Open the Project in VS Code
Open the newly created project in Visual Studio Code:
code .
Connecting to and Chatting with the AI Model
Now, modify Program.cs to integrate with Ollama.
Replace the contents of Program.cs with the following code:
using Microsoft.Extensions.AI;
IChatClient chatClient =
new OllamaChatClient(new Uri("http://localhost:11434/"), "phi3:mini");
// Store chat history
List chatHistory = new();
while (true)
{
// Get user input
Console.WriteLine("Your prompt:");
var userPrompt = Console.ReadLine();
chatHistory.Add(new ChatMessage(ChatRole.User, userPrompt));
// Get AI response
Console.WriteLine("AI Response:");
var response = "";
await foreach (var item in chatClient.GetStreamingResponseAsync(chatHistory))
{
Console.Write(item.Text);
response += item.Text;
}
chatHistory.Add(new ChatMessage(ChatRole.Assistant, response));
Console.WriteLine();
}
How the Code Works
- Initializes an
OllamaChatClientto interact with the AI model. - Maintains a conversation history for context-aware responses.
- Prompts the user for input and sends it to the AI model.
- Streams the AI’s response and displays it in real time.
Running the Chat Application
Start the application by running:
dotnet run
Example interaction:
Your prompt:
Tell me three facts about .NET.
AI Response:
Certainly! Here are three intriguing aspects of the Microsoft .NET framework:
1. **Language Interoperability** - One of the core strengths of .NET is its ability to support multiple programming languages through a common language runtime (CLR). This means that C#, F#, and other supported languages can interoperate seamlessly within the same project, allowing developers greater flexibility in tool selection.
2. **Managed Code** - The term 'managed code' refers to source code written for .NET which is compiled into Intermediate Language (IL). This IL then gets just-in-time compilation and runs on a virtual execution engine called the CLR. It offers advantages such as memory management, versioning control, exception handling, security features, etc., reducing manual coding errors considerably.
3. **Cross-Platform Support** - Though it originated with Windows, .NET has expanded to include cross-platform support for other operating systems like macOS and Linux since the introduction of .NET Core (now simply 'dotnet') in 2016. This means developers can create applications that work on multiple platforms without needing significant rewrites or changes in their codebase.
Shortening AI Responses
Since the AI stores conversation history, you can refine responses. Try entering:
Your prompt:
Shorten the length of each item in the previous response.The AI will now provide a concise summary.
Here’s your cleanup section formatted in HTML so you can add it to your existing blog:Cleaning Up Resources
Once you’ve completed the demo, follow these steps to clean up your system.
1️⃣ Stop the Ollama Server
Since Ollama runs in the background, stop it by running:
pkill -f "ollama serve"Or manually find the process and kill it:
ps aux | grep ollama
kill -9 <PID>
2️⃣ Remove the AI Model (Optional)
To free up disk space, remove the downloaded AI model:
ollama rm phi3:mini3️⃣ Delete the .NET Project
If you no longer need the project, navigate back and delete the directory:
cd ..
rm -rf LocalAI4️⃣ Uninstall Ollama (If No Longer Needed)
If you installed Ollama via Homebrew, remove it with:
brew uninstall ollamaIf you installed it manually, delete the application:
rm -rf /Applications/Ollama.app5️⃣ Remove .NET (Optional)
If you installed .NET for this demo and no longer need it, uninstall it:
brew uninstall dotnetOr, if installed manually, remove it with:
rm -rf /usr/local/share/dotnet
rm -rf ~/.dotnet
🎉 Cleanup Complete!
Your system is now clean and free of unused resources. 🚀
Conclusion
You’ve successfully built a .NET chat application that interacts with a local AI model. By using Ollama, you gain full control over AI execution while ensuring data privacy. The approach showcased here allows easy switching between different AI models with minimal changes.
Next Steps
- Experiment with different models from the Ollama Model Library.
- Enhance the app by integrating a graphical user interface.
- Deploy the application on a server for remote AI interactions.
