The Model Context Protocol (MCP) is quickly becoming the standard way to give AI models access to your tools and data. Think of it as a USB-C port for AI — a universal interface that lets any compatible LLM (Claude, GPT-4, Gemini) call your backend functions in a structured, safe, and consistent way.

But most tutorials stop at “run it locally.” This article shows you the full picture: build an MCP server in .NET, orchestrate it with .NET Aspire, and deploy it to Azure Container Apps — production-ready from the start.

End-to-end horizontal flow

The Stack at a Glance

GITHUB repo for this :

https://github.com/DileepSreepathi/learn-mcp/tree/master/sample-mcp

Project Structure

Three projects, clean separation of concerns. The AppHost never ships to production — it is the development and build-time orchestrator.

What Is MCP?

MCP is a JSON-RPC 2.0 protocol. When an AI model needs to call your tool, it sends a POST request like this:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "Add",
    "arguments": { "numberA": 5, "numberB": 3 }
  }
}

Your server executes the function and replies:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [{ "type": "text", "text": "8" }]
  }
}

Before calling tools, the model issues tools/list to discover what your server exposes and their input schemas. The ModelContextProtocol.AspNetCore library handles all of this wire-level plumbing for you.

Step 1 — Define Your Tools

This is where your business logic lives. The pattern is attribute-driven — decorate a class, decorate each method, done.

check this file path : //Calculator/Tools/CalculatorTool.cs

• [McpServerToolType] — tells the framework this class contains tools.

• [McpServerTool] — marks a method as an MCP-callable tool.

• [Description("...")] — this text is sent to the LLM as the tool’s description. Write it as you would document a public API.

• The framework uses reflection to automatically generate the JSON Schema for each tool’s parameters. Your C# types (double, long, List<long>) become the schema.

• Throwing an exception from a tool is safe — the framework catches it and returns a structured error to the client.

  ---

  Step 2 — Wire Up the MCP Server

  check this file : //Calculator/Program.cs


  Four things to notice:

  AddMcpServer() — registers the MCP protocol handler in the DI container.

  .WithHttpTransport(o => o.Stateless = true) — each HTTP request is fully independent. No session state. This is the right choice for containers and serverless platforms where any replica can serve any request.

  .WithTools<CalculatorTool>() — registers your tool class. The framework discovers all [McpServerTool] methods automatically.

  app.MapMcp("/mcp") — exposes the MCP endpoint. This single line is where LLMs connect to your server.

  ---

  Step 3 — Shared Infrastructure (ServiceDefaults)

  Rather than re-configure OpenTelemetry, health checks, and resilience in every service, Aspire projects use a ServiceDefaults shared library. One call, builder.AddServiceDefaults(), applies everything.


  check this path for the code :

  //ServiceDefaults/Extensions.cs

  • Structured logs with scopes and formatted messages — great for querying in Azure Log Analytics.

  • Distributed tracing — every HTTP call (including MCP tool calls) produces a trace span.

  • Runtime metrics — GC pressure, thread pool, memory.

  • HTTP resilience by default — any HttpClient automatically gets retry logic and a circuit breaker.

    And the health check endpoints

    public static WebApplication MapDefaultEndpoints(this WebApplication app)
    {
        if (app.Environment.IsDevelopment())
        {
            app.MapHealthChecks("/health");           // readiness
            app.MapHealthChecks("/alive", new HealthCheckOptions
            {
                Predicate = r => r.Tags.Contains("live")
            });                                        // liveness
        }
        return app;
    }
    
    

Step 4 — Aspire AppHost (Local Orchestration)

The AppHost is how you run everything locally without docker-compose or manual port juggling.

check here : //AppHost/Program.cs

When you dotnet run the AppHost:

1. It starts the Calculator service as a child process.

2. It assigns ports and injects connection strings/endpoints as environment variables.

3. It launches the Aspire Dashboard — a local UI showing logs, traces, and metrics for all services in real time.

WithExternalHttpEndpoints() tells Aspire (and Azure) that the service should be reachable from outside the container environment — not just from other services in the same mesh.

you can see the mcp server is running in the Aspire dashboard.

Run this npx @modelcontextprotocol/inspector at the /Calculator path , we connected to the mcp server and tools are listed where we can execute them.

Step 5 — Azure Deployment

The entire Azure deployment is configured in a single file:

# azure.yml
name: mcp-calculator
services:
  app:
    language: dotnet
    project: ./AppHost/AppHost.csproj
    host: containerapp

Run azd up and the Azure Developer CLI:

1. Reads azure.yml and uses the AppHost as the deployment manifest.

2. Builds a container image from the Calculator project.

3. Pushes the image to Azure Container Registry (ACR).

4. Creates a Container Apps Environment with managed HTTPS ingress.

5. Provisions a Log Analytics Workspace — OpenTelemetry telemetry flows here automatically.

6. Assigns a Managed Identity — no credentials, no secrets to rotate.

Once all the resources are deployed , you will get the url like :

Your MCP server is live at:

https://calc-mcp.proudsand-248b073a.eastus2.azurecontainerapps.io/mcp

connect the same on the npx inspector to access the tools.

What Makes This Production-Ready

Stateless HTTP transport — any container replica can handle any request. Scale out without sticky sessions.

OpenTelemetry from day one — traces, metrics, and logs flow to Log Analytics without any post-deployment wiring.

Managed Identity — the container accesses ACR and Azure resources using RBAC, not stored secrets.

Health probes — /alive (liveness) and /health (readiness) let the container platform restart unhealthy instances and remove them from rotation during startup.

Zero infrastructure code — Aspire generates the Container Apps configuration from the AppHost. No Bicep files to maintain manually.

The most important insight from this project is how little code you write to expose a meaningful MCP server:

• Tool implementation: ~60 lines of pure business logic

• MCP wiring: 5 lines in Program.cs

• Azure deployment: 6 lines of YAML

Everything else — the JSON-RPC protocol handling, JSON Schema generation, container packaging, HTTPS termination, health checks, and distributed tracing — is handled by the framework stack.

This is what the combination of ModelContextProtocol.AspNetCore, .NET Aspire, and azd is designed to do: let you focus on the capabilities you want to expose to AI, and handle the infrastructure plumbing as a solved problem.

Complete Request Life-cycle :

This is a conversation space exclusively for subscribers—kind of like a group chat or live hangout. I’ll post questions and updates that come my way, and you can jump into the discussion.

Join chat

How to get started

1. Get the Substack app by clicking this link or the button below. New chat threads won’t be sent sent via email, so turn on push notifications so you don’t miss conversation as it happens. You can also access chat on the web.

Get app

2. Open the app and tap the Chat icon. It looks like two bubbles in the bottom bar, and you’ll see a row for my chat inside.

3. That’s it! Jump into my thread to say hi, and if you have any issues, check out Substack’s FAQ.

Where We Left Off

In Part 1 we built a tiny MCP server that exposes a CalculatorTool.Add method.

In Part 2 we built an MCP client that connects to that server, discovers its tools, and calls them manually — you wrote CallToolAsync(”add”, {a:5, b:8}) explicitly in code.

That works, but it’s you doing the thinking. The whole point of MCP is to let an LLM do the thinking. This article wires them together: a user types natural language, the LLM decides which tool to call and with what arguments, and the result flows back into the model’s final answer.

Three things happen that didn’t happen in Part 2:

1. The LLM reads the tool descriptions and decides it needs to call add.

2. The UseFunctionInvocation() middleware intercepts the LLM’s tool-call request, executes the MCP tool, and feeds the result back to the LLM — all without you writing any loop.

3. The LLM composes the final answer from the tool result.

Prerequisites

Same as Part 2, plus one addition:

- .NET 10 SDK — [Download here](https://dotnet.microsoft.com/download)

- McpCalServer from Part 1 (already built)

- Ollama installed and running locally — [ollama.com](https://ollama.com)

- Pull the model: ollama pull llama3.2:latest

- Confirm it’s running: ollama serve (default port 11434)

> Why Ollama? It exposes an OpenAI-compatible REST API at http://localhost:11434/v1, so you can use the standard OpenAI client — no API key, no cloud, no cost. Swap it for Azure OpenAI or OpenAI proper by changing one URL.

Step 1: Create the Project

dotnet new console -n llm-client
cd llm-client

Install the packages:

dotnet add package Microsoft.Extensions.AI --prerelease
dotnet add package Microsoft.Extensions.AI.OpenAI --prerelease
dotnet add package ModelContextProtocol --prerelease
dotnet add package Microsoft.Extensions.Hosting

Your .csproj will look like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <RootNamespace>llm_client</RootNamespace>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>


 <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.AI" Version="10.*-*" />
    <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="10.*-*" />
    <PackageReference Include="Microsoft.Extensions.Hosting" Version="9.*-*" />
    <PackageReference Include="ModelContextProtocol" Version="0.*-*" />
  </ItemGroup>
</Project>

Two package families are in play here:

ModelContextProtocol : Speaks MCP — connects to the server, lists tools, calls them

Microsoft.Extensions.AI : .OpenAI Speaks to LLMs — abstracts chat clients, handles function-calling loops

Step 2: Write the LLM Client

Replace Program.cs with:

using Microsoft.Extensions.AI;
using ModelContextProtocol.Client;
using ModelContextProtocol.Protocol;
using OpenAI;
using System.ClientModel;
using System.Text.Json;

// Connect to Ollama via its OpenAI-compatible API
var openAiClient = new OpenAIClient(new ApiKeyCredential("ollama"), new OpenAIClientOptions
{
    Endpoint = new Uri("http://localhost:11434/v1")
});

IChatClient client = new ChatClientBuilder(openAiClient.GetChatClient("llama3.2:latest").AsIChatClient())
    .UseFunctionInvocation()
    .Build();

var clientTransport = new StdioClientTransport(new()
{
         Name = "Calculator Server",
        Command = "dotnet",
        Arguments = ["run", "--project", "../McpCalServer.csproj"]
});

Console.WriteLine("Setting up stdio transport");

await using var mcpClient = await McpClient.CreateAsync(clientTransport);

Console.WriteLine("Listing tools");
var mcpTools = await mcpClient.ListToolsAsync();
foreach (var tool in mcpTools)
{
    Console.WriteLine($"Connected to server with tools: {tool.Name}");
    Console.WriteLine($"Tool description: {tool.Description}");
}

// Use MCP tools directly — McpClientTool implements AITool
var chatOptions = new ChatOptions
{
    Tools = [.. mcpTools]
};

var userMessage = "add 2 and 4";
Console.WriteLine($"User: {userMessage}");

var response = await client.GetResponseAsync(userMessage, chatOptions);

// Check if any tool calls were made
foreach (var message in response.Messages)
{
    foreach (var content in message.Contents)
    {
        if (content is FunctionCallContent functionCall)
        {
            Console.WriteLine($"[TOOL CALLED] {functionCall.Name}({string.Join(", ", functionCall.Arguments?.Select(a => $"{a.Key}={a.Value}") ?? [])})");
        }
        else if (content is FunctionResultContent functionResult)
        {
            Console.WriteLine($"[TOOL RESULT] {functionResult.Result}");
        }
    }
}

Console.WriteLine($"Assistant: {response.Text}");

Step 3: Run It

dotnet run

Deep Dive: What’s Actually Happening

The Key Insight: McpClientTool Is an AITool

var chatOptions = new ChatOptions
{
    Tools = [.. mcpTools]
};

mcpTools is IList<McpClientTool>. The McpClientTool type (from the MCP SDK) implements AITool (from Microsoft.Extensions.AI). That’s not an accident — it’s the bridge the two ecosystems share.

When you spread mcpTools into ChatOptions.Tools, the LLM receives the tool’s name, description, and JSON input schema — exactly what it needs to reason about when and how to call it. No manual translation, no custom adapters.

The Agentic Loop: UseFunctionInvocation()

This single line is doing more work than it looks like:

IChatClient client = new ChatClientBuilder(...)
    .UseFunctionInvocation()  // ← this
    .Build();

Without it, when the LLM decides to call a tool, GetResponseAsync returns immediately and you get a raw FunctionCallContent in the response — you’d have to execute the tool yourself and call the LLM again. That’s the manual loop from Part 2, applied to the LLM side.

UseFunctionInvocation() installs a middleware that handles that loop automatically:

Reading the Message History

foreach (var message in response.Messages)
{
    foreach (var content in message.Contents)
    {
        if (content is FunctionCallContent functionCall) { ... }
        else if (content is FunctionResultContent functionResult) { ... }
    }
}

response.Messages is the full turn history added during this call — including intermediate tool calls and results. This is useful for:

- Debugging: see exactly which tool the LLM chose and with what arguments.

- Audit trails: log every tool invocation without modifying the tool itself.

- Multi-step reasoning: detect if the LLM called multiple tools in sequence.

response.Text is the convenience shortcut for the final assistant text.

Key Takeaways:

What Why it matters

McpClientTool implements AITool, Zero-glue bridge between MCP and Microsoft.Extensions.AI — spread MCP tools directly into ChatOptions

UseFunctionInvocation() middleware Handles the LLM → tool → LLM loop automatically; you await once

IChatClient abstraction Swap Ollama for Azure OpenAI or Claude

response.Messages Full audit trail of every tool call and result the LLM made

response.Text Final natural-language answer, ready to display

Stdio transport unchanged The LLM layer sits above MCP — the server doesn’t know or care there’s an LLM involved

What is an MCP Client?

An MCP client is the piece of code that connects to an MCP server , asks “what can you do?”, and then calls those capabilities on behalf of your application (or an LLM).

The entire client flow in our demo is four steps:

1. Transport → how to reach the server

2. Connect → handshake and negotiate

3. Discover → list tools (and resources)

4. Invoke → call a tool, get a result

Step 1 — Transport: how to reach the server

var clientTransport = new StdioClientTransport(new()
{
    Name = "Calculator Server",
    Command = "dotnet",
    Arguments = ["run", "--project", ".../McpCalServer.csproj"]
});

The transport answers one question: how do client and server talk?

“StdioClientTransport” means the client spawns the server as a child process and communicates via stdin/stdout. No ports, no network — just two processes piping JSON back and forth.

This is the standard pattern for local MCP servers. For remote servers you’d swap in an HTTP/SSE transport; everything else stays the same.

Step 2 — Connect: handshake and negotiate

await using var mcpClient = await McpClient.CreateAsync(clientTransport);

This single line does three things under the hood:

1. Starts the server process (via the transport)

2. Sends “initialize” — client tells the server its protocol version and capabilities

3. Receives server capabilities — does it support tools? resources? prompts?

After “CreateAsync” returns, both sides know what’s on the table. The await using ensures the connection (and child process) is cleaned up when done.

Step 3 — Discover: list tools and resources

var tools = await mcpClient.ListToolsAsync();
foreach (var tool in tools)
    Console.WriteLine($"  - {tool.Name}: {tool.Description}");

“ListToolsAsync” fires a `tools/list` request. The server responds with each tool’s name, description, and input schema (a JSON Schema object). Nothing is hard-coded — the client learns what exists at runtime.

This is the same data you’d pass to an LLM’s function-calling API. MCP tool schemas are natively compatible with OpenAI, Anthropic, and Gemini tool-use formats.

// Resources are optional — not all servers expose them
try
{
    var resources = await mcpClient.ListResourcesAsync();
}
catch (Exception ex)
{
    Console.WriteLine("No resources available: " + ex.Message);
}

Resources are data sources (files, DB rows, API responses) the server can expose. The `try/catch` isn’t sloppy code — it’s correct MCP behavior. If the server didn’t advertise resources during the handshake, calling `ListResourcesAsync` will throw. The client probes; the server declares he truth.

Step 4 — Invoke: call a tool, get a result

var addResult = await mcpClient.CallToolAsync(
    "add",
    new Dictionary<string, object?> { ["a"] = 5, ["b"] = 8 },
    cancellationToken: CancellationToken.None
);

“CallToolAsync” sends a `tools/call` request with the tool name and its arguments. The server runs the method and returns a content array — a structured result that can contain text, images, or embedded resources.

The `ExtractTextResult` helper in the demo pulls the first text item out of that array. In a real LLM agent you’d pass the whole content array back to the model ae tool result.it is shown in the below complete program.cs code.

Where to take it next:

1. Plug it into an LLM. Pass `tools` straight into `Microsoft.Extensions.AI`’s function-calling pipeline so a model can decide when to call `add` vs `help`.

2. Connect to multiple servers. Create one “McpClient” per server and merge their tool lists — the host doesn’t care where a tool lives.

3. Switch transports. Replace “StdioClientTransport” with an HTTP/SSE transport to talk to a remote MCP server (e.g. one hosted on Azure Container Apps).

4. Add resources and prompts. Extend the server with [McpServerResource] and [McpServerPrompt] to feel the full surface area.

5. Add auth + cancellation. Real clients pass `CancellationToken`s through (we already o) and layer auth on the transport.

complete program.cs code

using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using ModelContextProtocol.Client;
using System.Text.Json;



Console.WriteLine("🚀 Starting MCP C# Client...");

try
{
    var builder = Host.CreateApplicationBuilder(args);

        builder.Configuration
        .AddEnvironmentVariables()
        .AddUserSecrets<Program>();

    var clientTransport = new StdioClientTransport(new()
    {
        Name = "Calculator Server",
        Command = "dotnet",
        Arguments = ["run", "--project", "../McpCalServer.csproj"]
    });

     Console.WriteLine("📡 Connecting to MCP server...");

    // Create and connect the MCP client
    await using var mcpClient = await McpClient.CreateAsync(clientTransport);

    Console.WriteLine("✅ Connected to MCP server successfully!");

     // List available tools
    Console.WriteLine("\n📋 Listing available tools:");
    var tools = await mcpClient.ListToolsAsync();
    foreach (var tool in tools)
    {
        Console.WriteLine($"  - {tool.Name}: {tool.Description}");
    }
    // Test calculator operations
    Console.WriteLine("\n🧮 Testing Calculator Operations:");

     // Addition
    var addResult = await mcpClient.CallToolAsync(
        "add",
        new Dictionary<string, object?>() { ["a"] = 5, ["b"] = 8 },
        cancellationToken: CancellationToken.None
    );
    Console.WriteLine($"Add 5 + 8 = {ExtractTextResult(addResult)}");

     // Help
    var helpResult = await mcpClient.CallToolAsync(
        "help",
        new Dictionary<string, object?>(),
        cancellationToken: CancellationToken.None
    );
    Console.WriteLine($"\n📖 Help Information:");
    Console.WriteLine(ExtractTextResult(helpResult));

     // List resources if available
    try
    {
        Console.WriteLine("\n📄 Listing available resources:");
        var resources = await mcpClient.ListResourcesAsync();
        foreach (var resource in resources)
        {
            Console.WriteLine($"  - {resource.Name}: {resource.Description}");
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine("  No resources available or error listing resources: " + ex.Message);
    }

    Console.WriteLine("\n✨ Client operations completed successfully!");
}
catch(Exception ex)
{
     Console.WriteLine($"❌ Error running MCP client: {ex.Message}");
    Console.WriteLine($"Stack trace: {ex.StackTrace}");
}

static string ExtractTextResult(object result)
{
    try
    {
        if (result is IEnumerable<object> contentList)
        {
            foreach (var content in contentList)
            {
                if (content is IDictionary<string, object> contentDict &&
                    contentDict.TryGetValue("text", out var text))
                {
                    return text?.ToString() ?? "No text content";
                }
            }
        }
        
        // Fallback: try to serialize the entire result
        return JsonSerializer.Serialize(result, new JsonSerializerOptions { WriteIndented = true });
    }
    catch
    {
        return result?.ToString() ?? "No result";
    }
}

It’s one of the most misunderstood aspects of C# — and the answer might surprise you.

Every C# developer encounters this moment of confusion:

“Wait… if structs are value types that live on the stack, why am I using new to create them? Isn’t new for heap allocation?”

Let’s clear it up once and for all.

The Misconception

Many developers carry this mental model from languages like C or C++:

new = heap allocation = reference types only

In C#, this is incorrect. The new keyword doesn’t mean “allocate on the heap.” It means “call the constructor to initialize this object.” Where the memory lives is a completely separate concern.

Classes vs Structs: A Quick Refresher

// Reference type - lives on the heap
public class EmployeeClass
{
public string Name;
public int Id;
}

// Value type - lives on the stack
public struct EmployeeStruct
{
public string Name;
public int Id;
}

Here’s how they compare:

Class:

- Type category: Reference type

- Memory: Heap

- Default value: null

- Assignment: Copies the reference

Struct:

- Type category: Value type

- Memory: Stack (or inline within the containing object)

- Default value: All fields zeroed

- Assignment: Copies the entire value

What new Actually Does

Here’s the key insight — new behaves differently depending on the type:

For a Class → Allocates memory on the heap + calls constructor

For a Struct → Calls constructor only — no heap allocation

For structs, the memory already exists on the stack the moment you declare the variable. new simply initializes it.

You Don’t Need new for Structs

This is the part that surprises most people. Consider this struct:

public struct Point
{
public int X;
public int Y;
}

You can absolutely use it without new:

Point p; // memory is allocated on the stack right here
p.X = 10; // assign each field manually
p.Y = 20;
Console.WriteLine($"({p.X}, {p.Y})"); // (10, 20)

No new. No heap. It just works.

So Why Do People Use new With Structs?

Convenience. Without new, the compiler enforces a strict rule: you must assign every field before you read the struct.

Point p;
Console.WriteLine(p.X); // Compile error: Use of unassigned variable ‘p’
Even assigning some fields isn’t enough:
Point p;
p.X = 10;
Console.WriteLine(p.X); // Compile error: ‘p’ is not fully assigned

You must assign all fields:

Point p;
p.X = 10;
p.Y = 20;
Console.WriteLine(p.X); // Now it compiles

Using new zero-initializes everything in one shot:

Point p = new Point(); // X = 0, Y = 0 - fully assigned, ready to use
Console.WriteLine(p.X); // Works immediately

That’s it. That’s the reason. new on a struct is a shortcut for “set all fields to their default values.”

With Custom Constructors

Structs can have parameterized constructors just like classes:

public struct Point
{
public int X;
public int Y;
public Point(int x, int y)
{
X = x;
Y = y;
}
}
Point p = new Point(10, 20); // calls constructor - still NO heap allocation

The new here calls your constructor, but the Point still lives on the stack.

Proof: Visualizing Memory

void Demo()
{
Point s = new Point(5, 10); // struct - value type
EmployeeClass c = new EmployeeClass(); // class - reference type
}

Here’s what memory actually looks like:

The struct s lives entirely on the stack. The class c has a reference on the stack pointing to the object on the heap.

Classes MUST Use new

Unlike structs, you cannot use a class without new:

// Struct — valid without new
Point p;
p.X = 10;
p.Y = 20;  // ✓

// Class — NOT valid without new
EmployeeClass e;
e.Name = “Kumar”;  // ❌ Compile error — no object exists yet

A class variable without new is just a null reference. There’s no object in memory to write to. new is mandatory because it creates the object on the heap.

Assignment Behavior: The Real Difference

This is where the value-type vs reference-type distinction truly matters.

Struct (value type) — copies the data:

Point a = new Point(10, 20);
Point b = a; // copies all values into b
b.X = 99;
Console.WriteLine(a.X); // 10 - unchanged
Console.WriteLine(b.X); // 99 - independent copy

Class (reference type) — copies the reference:

EmployeeClass a = new EmployeeClass { Name = “Alice” };
EmployeeClass b = a;   // copies the REFERENCE — both point to same object

b.Name = “Bob”;

Console.WriteLine(a.Name);  // Bob  ← changed!
Console.WriteLine(b.Name);  // Bob  ← same object

Common Interview Question

“When should you use a struct instead of a class?”

Use a struct when:

- The type is small (Microsoft recommends under 16 bytes)

- It’s logically a single value (like Point, Color, DateTime)

- It’s immutable or short-lived

- You want to avoid heap allocations for performance

Use a class for everything else.

The new keyword in C# is not about where memory is allocated. It’s about calling a constructor to initialize an object. Structs already have their memory on the stack — new just makes sure all fields start with known values.

Next time someone asks why structs use new, you’ll know: it’s not allocation, it’s initialization.

. . .

Found this helpful? Give it a clap and share it with a fellow .NET developer who’s been confused by this too.

Follow me for more C# and .NET deep dives.

In the previous articles, you learned about the core concepts of the Model Context Protocol (MCP) — what it is, how it works, and why it matters. If not, check here:

In this article, I’m going to walk you through building a simple MCP server and setting up its environment from scratch. By the end, you’ll have a working MCP server that exposes a calculator tool that any MCP-compatible host (like Claude Desktop, VS Code Copilot, or your own client) can invoke.

What Are We Building?

A simple **MCP Server** that exposes a **Calculator Tool**. This tool will accept two numbers and return their sum. It’s minimal by design — the goal is to understand the structure and workflow, not to build something complex.

Prerequisites:

• - .NET 10 SDK (or later) installed

• IDE/Editor: Visual Studio Code.

• Package Managers: NuGet

• API Keys: For any AI services you plan to use in your host applications

Basic MCP Server Structure

An MCP server typically includes:

• Server Configuration: Setup port, authentication, and other settings

• Resources: Data and context made available to LLMs

• Tools: Functionality that models can invoke

• Prompts: Templates for generating or structuring text

For this demo, we’ll focus on Server Configuration and Tools.

Step 1: Create a .NET Console Application

dotnet new console -n MCPCalServer

cd MCPCalServer

This scaffolds a minimal .NET console application.

Step 2: Install the NuGet Packages

dotnet add package ModelContextProtocol --prerelease

dotnet add package Microsoft.Extensions.Hosting

Your `.csproj` file should now look something lie this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Hosting" Version="10.0.6" />
    <PackageReference Include="ModelContextProtocol" Version="1.2.0" />
  </ItemGroup>

</Project>

Replace the contents of `Program.cs` with the following:

using System.ComponentModel;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using ModelContextProtocol.Server;


var builder = Host.CreateApplicationBuilder(args);
builder.Logging.AddConsole(
    consoleLogOptions =>
    {
        consoleLogOptions.LogToStandardErrorThreshold = LogLevel.Trace;
    }
);

builder.Services
        .AddMcpServer()
        .WithStdioServerTransport()
        .WithToolsFromAssembly();
await builder.Build().RunAsync();


[McpServerToolType]
public static class CalculatorTool
{
    [McpServerTool, Description("Add two numbers")]
    public static string Add(int a , int b) => $"Sum {a +b}";
}

Let’s Break This Down

1. Host Setup

var builder = Host.CreateApplicationBuilder(args);

We use the .NET Generic Host (`Microsoft.Extensions.Hosting`) to create a long-lived application. This gives us dependency injection, configuration, and graceful shutdown out of the box.

2. Logging Configuration

builder.Logging.AddConsole(

consoleLogOptions =>

{

consoleLogOptions.LogToStandardErrorThreshold = LogLevel.Trace;

}

);

We configure console logging to write to **stderr**. This is important because MCP uses **stdio** (stdin/stdout) as the transport layer. If logs went to stdout, they would corrupt the MCP protocol messages. By directing logs to stderr, we keep the communication channel clean.

3. MCP Server Registration

builder.Services

.AddMcpServer()

.WithStdioServerTransport()

.WithToolsFromAssembly();

This is where the magic happens:

AddMcpServer()— Registers the MCP server into the dependency injection container.

WithStdioServerTransport()— Configures the server to communicate over standard input/output. This is the most common transport for local MCP servers.

WithToolsFromAssembly()— Automatically discovers and registers all classes marked with `[McpServerToolType]` in the current assembly. No manual registration needed.

4. The Calculator Tool

[McpServerToolType]

public static class CalculatorTool

{

[McpServerTool, Description(”Add two numbers”)]

public static string Add(int a, int b) => $”Sum {a + b}”;

}

[McpServerToolType]— Marks the class as containing MCP tools.

[McpServerTool]— Marks the method as an invocable tool.

Description(”Add two numbers”)— This description is sent to the LLM, helping it understand when and how to use the tool. Think of it as the tool’s documentation for the AI.

- The method parameters (`int a`, `int b`) are automatically exposed as the tool’s input schema. The MCP SDK handles serialization and deserialization for you.

Step 4: Test with MCP Inspector

MCP provides an Inspector tool that lets you interact with your server directly — without needing a full AI host application. It’s invaluable for development and debugging.

Run the following command from your project directory:

npx @modelcontextprotocol/inspector dotnet run

This will:

1. Start your MCP server (`dotnet run`)

2. Launch the MCP Inspector web UI

3. Connect the inspector to your server over stdio

Using the Inspector

Once the inspector is running, you should see your server’s tools listed. Here’s what to expect:

1. **Connect** to the server by clicking the “Connect” button.

2. Navigate to the **Tools** tab — you’ll see `Add` listed with its description “Add two numbers”.

3. Click on the `Add` tool to expand it.

4. Provide input values for `a` and `b` (e.g., a = 6 , b = 8).

5. Click **Run Tool** — you’ll see the output: `Sm 14`.

What’s Happening Under the Hood?

When you run the inspector, here’s the flow:

1. The **MCP Inspector** (client) spawns your server process and connects via **stdio**.

2. The client sends an `initialize` request, and the server responds with its capabilities (tools, resources, prompts).

3. When you invoke a tool from the inspector, it sends a `tools/call` request with the tool name and arguments.

4. The server routes the request to the matching method (`CalculatorTool.Add`), executes it, and returns the result.

5. The inspector displays the result.

This is the exact same protocol flow that happens when an LLM host like Claude Desktop or VS Code Copilot calls your tool.

What’s Next?

Congratulations on building your first MCP server! In the upcoming articles, we’ll explore:

- Adding more complex tools with multiple operations

- Exposing **Resources** to provide context to LLMs

- Using **Prompts** for structured text generation

- Connecting this server to a real AI host application

Thanks for following along — happy building!

What is MCP and why should you care?

If you’ve been building software for any meaningful amount of time, you’ve watched the industry cycle through integration standards — SOAP, REST, GraphQL, gRPC. Each one solved a real problem: how do two systems talk to each other in a structured, predictable way?

The Model Context Protocol (MCP) is the latest entry in that lineage, but it solves a fundamentally different problem. MCP isn’t about how your frontend talks to your backend. It’s about how AI models talk to tools, data sources, and external systems.

Think about it this way: when you use an AI assistant like Claude, it’s incredibly good at reasoning and generating text. But it’s trapped inside its own context window. It can’t check your calendar, query your database, search the web, or trigger a deployment pipeline — unless someone gives it a structured way to do those things. That structured way is MCP.

MCP is an open protocol — originally developed by Anthropic, but designed to be vendor-neutral and community-driven. It defines a standard way for AI models (the “client” side) to discover, negotiate with, and invoke capabilities exposed by external systems (the “server” side).

The problem MCP solves

Before MCP, every AI tool integration was bespoke. If you wanted Claude to interact with GitHub, you’d build a custom plugin. If you wanted it to query Postgres, you’d build another one. Each integration had its own message format, its own authentication story, its own error handling. It was like the pre-REST era — a mess of point-to-point integrations that didn’t compose.

The consequences were painful:

For tool builders, you had to write separate integrations for every AI provider — one for OpenAI’s function calling format, another for Claude’s tool use format, another for Gemini. The same tool, reimplemented three times.

For AI application developers, you couldn’t mix and match. Switching AI providers meant rewriting all your tool integrations. You were locked in — not by the model itself, but by the plumbing around it.

For enterprises, the security story was a nightmare. Each custom integration had its own trust model. There was no standardized way to enforce consent, rate limiting, or data flow controls across all AI-tool interactions.

MCP solves all three of these by establishing a single, open protocol that any client and any server can implement. Build your tool once as an MCP server, and it works with every MCP-compatible AI client. Switch your AI provider, and all your tools keep working. Enforce your security policies in one place, and they apply to every integration.

The two-layer architecture

MCP's architecture is elegantly simple. The entire protocol is split into exactly two layers, each with a clearly defined responsibility. If you've ever worked with the OSI model or even just the separation between ASP.NET's middleware pipeline (how requests are processed) and Kestrel (how bytes move over the wire), this pattern will feel immediately familiar.

The Data Layer — what to say

The Data Layer is the brain of the protocol. It defines what clients and servers say to each other — the structure of messages, the semantics of each operation, and the rules governing the conversation lifecycle.

It’s built on top of JSON-RPC 2.0, which is a lightweight remote procedure call protocol that uses JSON as its data format. If you’ve never encountered JSON-RPC before, think of it as “HTTP endpoints, but instead of URLs and verbs, you have a method string and params object.”

The Data Layer handles three core responsibilities: defining the message structure (request, response, notification), managing the connection lifecycle (initialization, capability negotiation, shutdown), and exposing protocol primitives (tools, resources, and prompts — the things an MCP server can offer).

The Transport Layer — how to deliver it

The Transport Layer is the postal service. It doesn’t care about what’s inside the envelope — it just makes sure the envelope gets from point A to point B reliably.

MCP supports two transport mechanisms:

Stdio (Standard I/O) is used for local communication. The MCP client spawns the MCP server as a child process and they talk through stdin/stdout pipes. This is fast, requires zero network configuration, and is the default for local development tools. I

Streamable HTTP with Server-Sent Events (SSE) is used for remote communication. The client sends messages to the server via HTTP POST requests, and the server can push real-time updates back via SSE. This is the transport you’d use when your MCP server is running on a different machine — say, as a cloud service.

As a engineer who’s spent years building APIs and distributed systems, MCP feels like a natural evolution. It takes patterns we already know — request/response messaging, service discovery, middleware pipelines — and adapts them for the AI agent era. The protocol is intentionally simple: JSON-RPC 2.0 messages flowing over either local stdio or remote HTTP, with a clean lifecycle of initialize → discover → execute.

In the world of Enterprise AI, a Large Language Model (LLM) is like a brilliant scholar who has read almost everything on the public internet but has never seen your company’s internal documents. If you ask it about your specific “Q3 Project Roadmap” or “Internal Remote Work Policy,” it might guess—or worse, hallucinate.

To bridge this gap, we use RAG (Retrieval-Augmented Generation): check here . Today, I’ll walk you through how to use Azure AI Search as the ultimate “librarian” for your LLM, specifically using C# and Azure AI Foundry.

Instead of trying to “teach” the LLM your data by retraining it (which is slow and expensive), we connect it to a data source.

By using a C# Console Application demo, we can orchestrate a flow where the LLM doesn’t just generate text; it first searches your private data (stored in Azure Blob Storage) via Azure AI Search and then uses that specific information to form an answer.

The Tech Stack: Azure Resources

To build this, we utilize three core pillars of the Azure ecosystem:

1. Azure Blob Storage: The “Warehouse” where your raw PDF files live.

2. Azure AI Search: The “Search Engine” that indexes, chunks, and vectorizes those PDFs.

• 3. Azure AI Foundry: The “Orchestrator” where we deploy the LLM and manage the connection between the model and the search index.

How RAG Actually Works

Attaching a data source isn’t just a simple “plugin.” Because LLMs have Token Limits (a maximum amount of text they can process at once), we can’t just send 100 PDFs into a single prompt. This is where the RAG pipeline shines:

1. Ingestion & Document Cracking

The Azure AI Search Indexer reaches into your Blob Storage. It “cracks” open your PDFs and extracts the raw text.

2. Chunking & Vectorization

This is the “Magic” step. To stay within token limits, the indexer breaks the text into Chunks (smaller pieces, like 500 words each).

• Each chunk is then sent to an Embedding Model.

• The model converts the text into a Vector (a long list of numbers representing the meaning of the text).

3. The Retrieval (The “R” in RAG)

When a user asks a question in your C# app, the app doesn’t send the question to the LLM yet. Instead:

• The question is turned into a vector.

• Azure AI Search finds the “top 3” chunks in your index that are mathematically closest to the question.

4. Generation (The “G” in RAG)

Your app sends a “Super-Prompt” to the LLM:

“Here is the context from our PDFs: [Chunk 1, Chunk 2]. Now, based ONLY on this context, answer the user’s question: [User Query].”

Why Use Vector Search?

Traditional keyword search looks for exact words. If you search for “Salary,” you might miss a document that says “Compensation.”

Vector Search understands intent. Because it operates on mathematical “meaning,” it knows that “Salary” and “Compensation” are the same thing. This makes your AI much more intuitive and “human-like” in how it finds information.

In the practical example below, Uploading a single PDF file to Azure Blob Storage and importing it to Azure AI Search with an indexes. The LLM model is deployed on Azure AI Foundry, accessed via API URL and KEY. We use the chat completion feature in Azure OpenAI SDK.

Blob Storage:

Azure AI Search:

Azure AI Foundry: using gpt-4.1 LLM model

Practical Demo :

using OpenAI.Chat;
using Azure;
using Azure.AI.OpenAI;
using Azure.AI.OpenAI.Chat;

#pragma warning disable AOAI001

// Azure OpenAI configuration
var endpoint = new Uri("https://ai-labs-project.cognitiveservices.azure.com/");
var deploymentName = "gpt-4.1";
var apiKey = "8Pio3V4acuKq6M1BbSbIJ50YQ3D8IIpacjLpPLs0TJJtqLamflYYJQQJ99CBACYeBjFXJ3w3AAAAACOGdij7";

// Azure AI Search configuration
var searchEndpoint = "https://ragpdf.search.windows.net";
var searchApiKey = "NJiH9F3ZQCwrCib3K7C5AMH2IJsF1ocJVoGcwcWjzWAzSeC6dTAR";
var searchIndexName = "ragskills";

AzureOpenAIClient azureClient = new(
    endpoint,
    new AzureKeyCredential(apiKey));

ChatClient chatClient = azureClient.GetChatClient(deploymentName);

var chatCompletionOptions = new ChatCompletionOptions();
chatCompletionOptions.AddDataSource(new AzureSearchChatDataSource()
{
    Endpoint = new Uri(searchEndpoint),
    IndexName = searchIndexName,
    Authentication = DataSourceAuthentication.FromApiKey(searchApiKey),
});

List<ChatMessage> messages = new List<ChatMessage>()
{
    new SystemChatMessage("You are a helpful assistant. Use the provided data source to answer questions."),
    new UserChatMessage("What skills are mentioned in the documents?"),
};

var response = chatClient.CompleteChat(messages, chatCompletionOptions);

// Print the main response
Console.WriteLine("Response:");
Console.WriteLine(response.Value.Content[0].Text);

Output: I have uploaded the PDF file containing the skills section, which is the same section we are querying.

By combining Azure AI Search with the power of LLMs in Azure AI Foundry, we move beyond generic AI. We create a specialized system that knows your business, respects your data privacy, and provides grounded, accurate answers.

Hello ,

In my previous article - check here, we discussed the creation of the agent and its interaction with the database and other tools.

In this article, I provide a comprehensive guide to constructing a **Custom Sales Agent**—an interactive .NET console application that enables users to pose natural-language inquiries regarding Contoso sales data. The agent’s functionalities include:

A practical guide to building an AI-powered sales assistant that runs locally with Ollama during development and scales to Azure AI Foundry in production — complete with LLM tool-calling, SQLite querying, and Terraform infrastructure-as-code.

Large Language Models (LLMs) possess remarkable capabilities in answering questions in natural language. However, they lack the ability to natively query private databases. **Tool calling** (also known as function calling) using the semantic kernel effectively addresses this limitation. The LLM determines the appropriate time to invoke your code and the SQL query to generate, while your application securely executes the query against the actual data.

Local Development with Ollama: Utilizing llama3.2 for expedient and cost-effective development cycles.

Production Deployment with Azure AI Foundry Agent Service: Transitioning to GPT-4.1 for production purposes.

Dynamic SQL Query Generation and Execution: Employing LLM tool calling to dynamically generate and execute SQL queries against a SQLite database.

Terraform-Based Azure Infrastructure Provisioning: Automating the provisioning of all Azure infrastructure.

Architecture flow :

Project Structure :

once the repo is cloned - connect with your Azure account and run the terraform init ,validate and then apply it - all its required resources are created on the Azure cloud.

use the below apply command:

terraform apply -var=”deploy_cloud_model=true” -auto-approve 2>&1 | tail -20

Development Mode:

install the Ollama model run the below commands and check the local Model running.

brew install ollama
ollama pull llama3.2

ollama serve

you can confirm by checking this weather the local model is running or not :

and then run in development mode: DOTNET_ENVIRONMENT=development dotnet run

You can now engage in natural language conversations with the agent, and it will provide responses in the same natural language, ensuring clarity and comprehension.

Production Mode:

before running the prod mode , make sure all the resources on the azure are created and verify the LLM model is present on the Azure Foundry.

and then run in prod mode: you can see the result below.

How Tool Calling Works

The big idea here is **LLM tool calling**. Instead of the LLM just trying to answer based on what it’s learned, it does this:

1. It gets the user’s question in plain language, along with the database’s structure.

2. It figures out that it needs some data and makes a tool call with a SQL query.

3. Our code steps in and runs the SQL against SQLite, then sends back the results.

4. The LLM then turns the raw data into a helpful, easy-to-understand answer.

This pattern ensures data privacy by preventing the LLM from storing it. It also guarantees that answers are always based on real data.

Understanding Semantic Kernel:

Before understanding Semantic Kernel, we need to understand the problem with just using a Large Language Model (LLM) like GPT-4 directly.

Imagine an LLM is a brilliant, world-renowned scholar locked in an empty room.

• They know everything about the public internet up to their training cut-off.

• They can write poems, solve complex math problems, and code in Python.

But this scholar has severe limitations:

1. They don’t know you: They have no access to your company’s private documents, databases, or emails.

2. They can’t do anything: They cannot send an email, book a meeting, query a SQL database, or push a button in Salesforce. They can only output text.

3. They have amnesia: Every new conversation starts with a blank slate. They don’t remember what you asked them five minutes ago unless you re-send the entire conversation history.

If you want to build a useful AI application (like a chatbot that can answer questions about your internal HR policies and then book time off for you), the “naked” LLM cannot do it alone.

You need a bridge. Semantic Kernel is the bridge here.

In technical terms, it is an open-source Software Development Kit (SDK) from Microsoft, available in C#, Python, and Java.

In simple terms, it is the “operating system” or the “orchestrator” for your AI application. It sits between your application code and the AI models.

The Core Concept: Everything is a “Function”

The most crucial aspect to comprehend about SK is its ability to blur the distinction between conventional code and AI prompts. It treats both as “functions” that can be interconnected.

1. Native Functions (Code): These are standard C# or Python functions that you write. They perform deterministic tasks, such as calculating taxes, querying databases, sending emails, and retrieving the current time.

2. Semantic Functions (Prompts): These are instructions written in natural language for the LLM. For instance, “Please summarize the provided input text into three concise bullet points.”

Building an AI agent that bridges natural language and structured data doesn’t require a massive framework. With .NET’s dependency injection, clean service abstraction, and tool-calling capabilities in both open-source (Ollama) and commercial (Azure AI) LLMs, you can create a powerful, production-ready agent in a few hundred lines of code.

The dual-provider pattern offers fast iteration cycles and enterprise-grade deployment: free local development with Ollama and scalable production with Azure AI Foundry.

The complete source code is available on [GitHub] https://github.com/DileepSreepathi/custom-sales-agent

You’re a seasoned software engineer, a decade or more deep in the trenches, shipping code, solving complex problems, and navigating the ever-shifting sands of technology. You’ve built systems from the ground up, refactored monoliths, danced with micro-services, and probably have a graveyard of forgotten frameworks in your mental attic.

But despite the countless lines of code written and systems launched, there’s a subtle yet profound distinction that often gets blurred, even by the most experienced among us: the line between Architecture and Design.

This isn’t just semantics. Understanding this divide – truly understanding it – isn’t about passing a certification. It’s about fundamentally altering how you approach problem-solving, how you build resilient systems, and ultimately, how you sculpt software that stands the test of time, scales under pressure, and evolves with grace. This article isn’t just going to define the terms; it’s going to explore why this distinction holds the key to your next decade of engineering excellence.

The Elephant in the Room: Why Do We Confuse Them?

Let’s be honest, the terms “architect” and “designer” are often used interchangeably in job titles and daily conversations. We talk about “designing an architecture” or “architecting a component.” This linguistic fuzziness reflects a deeper systemic issue: a lack of explicit differentiation in many organizations.

Part of the confusion stems from the historical evolution of our craft. In the early days, a single engineer often handled everything from the foundational structure to the minutiae of a class’s methods. As systems grew in complexity, specializations emerged. Yet, the mental model often lagged, leading to architects dabbling in low-level design and senior designers making architectural choices without a holistic view.

• Architectures that are beautiful on paper but impossible to implement.

• Designs that are elegant at a component level but create integration nightmares.

• Systems that scale horizontally but are brittle internally.

• Teams constantly refactoring the wrong things because the foundational principles were never clearly established.

Architecture: The Blueprint of Existence

Think of software architecture like the blueprint of a city or a large building. It defines the fundamental structure, the main components, their relationships, and the principles governing their interaction.

Key Characteristics of Architecture:

1. High-Level, Strategic Decisions: Architecture deals with the “what” and “why” at a macro level. It answers questions like:

   • What are the primary systems involved (e.g., microservices, monolith, serverless)?

   • How will they communicate (e.g., REST, gRPC, Kafka, message queues)?

   • What are the core technologies and frameworks (e.g., Kubernetes, .NET Core, Node.js)?

   • How will data be stored and accessed across the system (e.g., SQL, NoSQL, data lakes)?

   • What are the non-functional requirements (NFRs) that must be met (scalability, security, performance, fault tolerance, maintainability)?

   • How will deployment, monitoring, and logging be handled across the entire ecosystem?

2. Concerned with Cross-Cutting Concerns: Architectural decisions inherently impact multiple parts of the system. Security protocols, logging standards, error handling mechanisms, transaction management – these are architectural concerns that must be consistent across the board.

3. Shapes Constraints and Boundaries: Architecture defines the sandbox within which design can happen. It sets the rules, dictates the available tools, and outlines the integration points. For instance, if the architecture mandates an event-driven microservices approach, individual services must conform to that paradigm.

4. Long-Term Vision and Stability: Architectural decisions are notoriously difficult and expensive to change once implemented. They represent a significant investment and commitment. A good architecture anticipates future needs and provides a robust, adaptable foundation, minimizing the need for complete overhauls.

5. Focus on “ilities”: Scalability, reliability, maintainability, testability, security, usability, deployability – these “ilities” are the bread and butter of architecture. They are the quality attributes that ensure the system remains viable and effective over its lifespan.

Analogy: If your software is a house, the architecture defines whether it’s a skyscraper, a bungalow, or a tent. It determines the foundation, the number of floors, the main structural beams, the plumbing and electrical systems, and how these major systems interact. It ensures the building can withstand earthquakes (fault tolerance) and accommodate future expansions (scalability).

Design: The Art of Implementation and Detail

If architecture is the city plan, then design is the detailed blueprint of a specific building, a floor plan, or even an individual room. It’s about bringing the architectural vision to life, detailing how a specific component or system will fulfill its architectural mandate.

Key Characteristics of Design:

1. Detailed, Component-Level Decisions: Design deals with the “how” at a granular level. It answers questions like:

   • How will this specific microservice be structured internally (e.g., layered architecture, hexagonal architecture)?

   • What classes, interfaces, and modules will it contain?

   • How will data flow within this component?

   • What specific algorithms will be used to implement a feature?

   • How will error handling be implemented within this service?

   • What design patterns (e.g., Factory, Singleton, Observer) will be applied at a class or module level?

   • What are the specific API contracts for a service or a module?

2. Focus on Specific Functionality: Design is concerned with the implementation of particular features, functionalities, or services as defined by the architecture.

3. Within Architectural Constraints: Design decisions are made within the boundaries and guidelines established by the architecture. A designer cannot suddenly decide to use a different database for their microservice if the architecture dictates a specific data storage strategy.

4. More Iterative and Adaptable: While good design is also crucial, its scope is more localized. Changes to specific design elements are generally less impactful and less costly than changing fundamental architectural decisions. Design evolves as features are built and refined.

5. Concerned with Code Quality and Developer Experience: Good design leads to clean, maintainable, readable, and testable code. It directly impacts the day-to-day life of engineers working on the codebase. Principles like SOLID, DRY, YAGNI are paramount here.

Analogy: Continuing with the house analogy, design defines the layout of each room, where the walls go, the specific type of windows, the material of the floors, the placement of electrical outlets, and how the furniture will be arranged. It ensures each room is functional, aesthetically pleasing, and serves its purpose within the overall house structure.

Architecture and Design are not isolated silos. They are two sides of the same coin, deeply intertwined and mutually dependent.

• Architecture Informs Design: Architecture provides the overarching framework, the guiding principles, and the constraints that inform all subsequent design decisions. A robust architecture makes design simpler and more consistent.

• Design Validates Architecture: Detailed design and implementation can expose flaws or impracticalities in the architectural vision. Sometimes, what looks good on an architectural diagram proves challenging or inefficient to implement in practice, necessitating a feedback loop to the architectural layer.

• Poor Design Can Undermine Architecture: Even the most brilliant architecture can crumble under the weight of poor design. Inefficient algorithms, tightly coupled components, or a lack of attention to detail at the design level can negate all the benefits of a well-thought-out architecture.

• Tactical vs. Strategic: Architecture is inherently strategic; it looks years ahead. Design is more tactical, focused on delivering features within the current sprint or release cycle, but always within that strategic architectural context.

For us, the seasoned engineers, understanding this distinction is more than academic. It’s about elevating our craft and maximizing our impact:

1. Speak the Right Language: When discussing problems, be precise. Is it an architectural problem (e.g., “our system can’t handle the load at scale”), or a design problem (e.g., “this service’s internal module coupling is too high”)? This clarity streamlines communication and problem-solving.

2. Contribute at the Right Level: As senior engineers, you’re expected to contribute to both. When engaged in architectural discussions, focus on the big picture, the NFRs, and the long-term viability. When diving into a component, focus on elegance, maintainability, and efficiency.

3. Bridge the Gap: Often, architects create the vision, and individual teams execute the design. You, with your experience, are perfectly positioned to be the bridge. You can translate architectural mandates into actionable design principles for your team and provide valuable feedback from the trenches back to the architects.

4. Mentor the Next Generation: Teach junior engineers this distinction early. It instills a holistic view of software development that prevents them from getting lost in the weeds or making premature architectural decisions at the wrong level.

5. Architect for Evolvable Design: A truly great architecture doesn’t just work; it enables great design. It offers clear interfaces, defined boundaries, and a robust framework that allows designers to build flexible, high-quality components without constantly fighting the system.

Building for Tomorrow, Today

In our rapidly evolving industry, the allure of the next shiny framework can often overshadow the fundamental principles of software construction. But for those of us committed to building lasting, impactful software, the distinction between architecture and design is paramount.

Architecture lays the resilient foundation, defines the grand vision, and safeguards the strategic future of your system. Design then meticulously crafts the internal mechanisms, ensuring efficiency, maintainability, and immediate functionality.

When you understand these small but important differences, you become a better leader and decision-maker. You stop just “writing code” and start building complete environments. Instead of making something that only works right now, you create systems that are ready to grow and handle whatever happens in the future.

What are your experiences with this distinction? Have you seen its impact – positive or negative – in the systems you’ve built? Share your thoughts below!

If there is one lesson that elevates a developer from “Junior” to “Senior,” it’s learning to prefer Composition over Inheritance. The Strategy Pattern is the ultimate manifestation of this principle.

In this deep dive, we are going to look at a common scenario—building a car manufacturing system. We will see how a naive approach traps us in a web of dependencies (The “God Class”), and how the Strategy Pattern acts as a “Plugin System” to restore sanity, testability, and flexibility.

1. The Domain Scenario

Imagine we are building the software backbone for a futuristic automotive group. We have three manufacturers:

• Company A: Makes standard economy cars.

• Company B: Makes hybrids.

• Company C: An experimental division working on flying cars and AI.

The Challenge: We need to model these cars. Some fly, some don’t. Some use petrol, some use electric, some are hybrid. New features (like “Autonomous Mode”) are coming next year.

2. The Naive Approach (The Trap)

The instinctual approach is to use inheritance. We create a base Car class and shove every possible feature into it.

public abstract class Car
{

    // Core features

    public void Drive() { /* ... */ }

    // The Problem: Anticipating every future feature in the base class

    public virtual void Fly() { /* Default implementation? Empty? Throw Exception? */ }

    public virtual void ActivateHybrid() { /* ... */ }

    public virtual void AutonomousPark() { /* ... */ }

}

Why this fails (The “God Class” Anti-Pattern)

As soon as you create EconomyCar (which can’t fly) or VintageCar (which isn’t hybrid), you hit a wall.

1. Polluted Interface: EconomyCar technically has a .Fly() method. This is confusing for other developers using your code.

2. Liskov Substitution Violation: You might override Fly() to throw a NotImplementedException. Now, any code iterating through a list of Car objects and calling .Fly() risks crashing the application.

3. Fragile Base Class: If Company C needs a change to the Fly logic, you might edit the base class, accidentally breaking Company A‘s code.

3. The Solution: The Strategy Pattern

The Strategy Pattern suggests: Take what varies and encapsulate it.

Instead of the Car being the behavior, the Car has a behavior. We break the monolithic Car class into a host and a set of pluggable components.

Step 1: Define the Interfaces (The Sockets)

We identify the behaviors that vary: Flying and Powertrain.

// The contract for any flying behavior

public interface IFlyStrategy

{

    void Fly();

}

// The contract for any engine behavior

public interface IPowertrainStrategy

{

    void Drive();

}

Step 2: Implement the Strategies (The Plugs)

We create concrete classes for every variation. These are small, easy to test, and isolated.

// --- Flying Strategies ---

public class JetPropulsion : IFlyStrategy

{

    public void Fly() => Console.WriteLine(”🚀 Thrusters engaged! We are flying.”);

}

public class NoFly : IFlyStrategy

{

    public void Fly() => Console.WriteLine(”🚫 This car cannot fly.”);

}

// --- Powertrain Strategies ---

public class PetrolEngine : IPowertrainStrategy

{

    public void Drive() => Console.WriteLine(”⛽ Vroom! Driving on petrol.”);

}

public class ElectricEngine : IPowertrainStrategy

{

    public void Drive() => Console.WriteLine(”⚡ Silent but deadly. Driving on electric.”);

}

Step 3: The Context (The Car)

The Car class no longer implements the logic. It simply holds references to the interfaces and delegates the work.

public abstract class Car

{

    // Composition: The Car HAS strategies

    private IFlyStrategy _flyStrategy;

    private IPowertrainStrategy _powertrainStrategy;

    // We can inject strategies via constructor

    protected Car(IFlyStrategy flyStrategy, IPowertrainStrategy powertrainStrategy)

    {

        _flyStrategy = flyStrategy;

        _powertrainStrategy = powertrainStrategy;

    }

    // Dynamic Swapping: We can change behavior at runtime!

    public void SetFlyStrategy(IFlyStrategy newStrategy) => _flyStrategy = newStrategy;

    // Delegation: “I don’t know how to fly, I ask my strategy to do it.”

    public void PerformFly() => _flyStrategy.Fly();

    public void PerformDrive() => _powertrainStrategy.Drive();

}

4. Putting It Together

Now, creating specific car models is just a matter of “configuring” them with the right LEGO blocks.

// Company A: Standard Petrol Car

public class CityCommuter : Car

{

    public CityCommuter() : base(new NoFly(), new PetrolEngine()) { }

}

// Company C: Experimental Flying Car

public class SkyRacer : Car

{

    public SkyRacer() : base(new JetPropulsion(), new ElectricEngine()) { }

}

The “Aha!” Moment: Runtime Swapping

Because we rely on interfaces, we can change a car’s behavior while the program is running. Imagine a video game where a car picks up a “Flight Power-up”:

var myCar = new CityCommuter();

myCar.PerformFly(); // Output: “This car cannot fly.”

// PLAYER PICKS UP POWER-UP

Console.WriteLine(”--- UPGRADE APPLIED ---”);

myCar.SetFlyStrategy(new JetPropulsion());

myCar.PerformFly(); // Output: “Thrusters engaged! We are flying.”

5. Why This is “Clean Code” (SOLID Analysis)

The Strategy Pattern is not just about organizing code; it is about organizing change.

• Naive Code: Resists change. New features break old features.

• Strategy Code: Welcomes change. New features are just new classes waiting to be plugged in.

By decoupling the What (The Car) from the How (The Engine/Wings), you build a system that can evolve as fast as your business requirements.

refer the GITHUB repo for this pattern - the above implementation may differ https://github.com/DileepSreepathi/StrategyDesignPattern

Before learning and implementing agents in the .NET ecosystem, let’s understand some basics. I suggest reading this to get some understanding of ML.Net, just for an idea to understand some basics of LLMs as well

The Foundation: What is a Large Language Model (LLM)?

Before we get to the future, let’s ground ourselves in the present. At its core, an LLM is a sophisticated text-prediction engine.

Think of it as an incredibly well-read consultant. It has consumed a vast portion of the internet, books, and articles. You can ask it to:

• Generate content (write a blog post, a marketing email, or a programming language script).

• Summarize a long document.

• Translate between languages.

• Answer a complex question.

You give it a prompt (input), and it gives you a response (output).

But here’s the key limitation: An LLM is passive. It can’t do anything outside of its text box. It can tell you how to book a flight, but it can’t open a browser, go to the airline’s website, and actually book it for you. It’s all talk, no action.

The Evolution: Meet the AI Agent

An AI Agent is what you get when you take that brilliant LLM “brain” and build an autonomous system around it. An agent isn’t just a responder; it’s a doer.

If an LLM is a consultant who gives you a plan, an AI Agent is the project manager who takes your goal and executes the entire plan, end-to-end.

An agent is typically composed of four key parts:

1. The LLM (The “Brain”): The agent uses the LLM for its core reasoning, planning, and decision-making. “The user wants to book a flight. What’s the first step?”

2. Tools (The “Hands”): This is the game-changer. An agent has access to a set of tools (like APIs) that allow it to interact with the real world. These tools can include:

   • Web or file search.

   • Code execution (to run a script).

   • Database access (to check your stored preferences).

   • APIs (to connect to your calendar, your email, or the airline’s booking system).

3. Planning & Task Decomposition: When you give an agent a complex goal (e.g., “Plan my weekend trip to Cape Town”), it uses its LLM brain to break that goal down into a series of steps:

   • Step 1: Search for flights to Cape Town for this Friday.

   • Step 2: Check my calendar for conflicts.

   • Step 3: Find top-rated hotels near the waterfront.

   • Step 4: Present the top 3 options to the user.

   • Step 5: Await approval and then book the chosen option.

4. Memory (The “Scratchpad”): An agent can remember the results of its previous actions. It can learn from its mistakes and self-correct. If a search fails, it can try a different query. This “inner monologue” or “chain of thought” allows it to handle multi-step tasks without human intervention.

In this post, we will explore and implement a code-first ‘Sales Agent’ designed to analyze sales data. This is an LLM-powered agent, built using the Semantic Kernel framework, which means it uses an LLM to process requests. It connects to external tools like databases, hosted files, and a code interpreter to generate charts.

Here is the high-level workflow for the LLM-powered ‘Sales Agent’.

Now that we have a foundational understanding of what LLMs and agents are, the next logical step is orchestration. To effectively orchestrate and maintain these agents, a dedicated platform is required. This is precisely where Azure AI Foundry comes into the picture.

Azure AI Foundry is a managed service from Microsoft on the Azure cloud designed to handle the entire agent lifecycle. It provides essential capabilities, such as:

• Rapid deployments and automation

• Secure data connections

• Flexible model selection

• Enterprise-grade security

What is Semantic Kernel? Your AI Orchestra Conductor

If you’re a developer looking to build sophisticated AI applications, you’ve probably hit a wall. Large Language Models (LLMs) like those from Azure OpenAI are incredibly powerful at understanding and generating language, but they live in their own world. They can’t browse your company’s database, send an email, or call your existing C# functions.

This is where Semantic Kernel framework (SK) comes in.

Think of Semantic Kernel as an AI orchestra conductor. Your LLM (like GPT-4) is the star musician—brilliant, but it only knows how to play its instrument (language). Your traditional code (like your .NET business logic, APIs, and databases) are the other musicians in the orchestra, each with a specific skill.

Semantic Kernel is the conductor that hands the right sheet music (the prompt) to the LLM and coordinates it with the other musicians (your code) to create a complex, harmonious piece of music (a useful application).

It is an open-source SDK from Microsoft that acts as a lightweight orchestration layer, allowing you to seamlessly blend AI with conventional programming languages like C#, Python, and Java.

In short:

• Semantic Kernel is the open-source SDK (the “how-to”) for building AI agents.

• Azure AI Studio / Foundry is the managed platform (the “where-to”) to deploy, run, and scale those agents in the cloud.

For example, the framework provides a specific AzureAIAgent class. This agent is designed to run within the Azure AI service, seamlessly connecting to your Azure AI Search for data, using your Azure OpenAI models, and automatically handling things like conversation history and tool (plugin) execution.

Traditional applications operate on predefined logic. When you pass instructions, the application follows a fixed path to produce a result that is already planned and structured. The output is predictable because the entire process is hard-coded.

In contrast, LLM-powered agents operate dynamically. When you provide instructions and context, the agent itself intelligently determines the best way to handle the task, rather than following a rigid script. This demonstrates their power and speed, as they can significantly reduce manual effort by adapting to the request. It’s a much more intelligent and flexible approach.

Azure AI Foundry is designed to change that. It’s a platform that combines models, tools, frameworks, and governance into a unified system for building intelligent agents. At the center of this system is Azure AI Foundry Agent Service, enabling the operation of agents across development, deployment, and production.

Let’s jump to coding part :

Github Link : https://github.com/DileepSreepathi/sales-agent-workshop

Resources setup: once the project is cloned , try to set up the resources that are required in this project. The required resources are created in the Azure cloud by running the below command.

./deploy.sh

After the Azure resources are deployed, open the project workspace in VS Code and run the application (e.g., by typing dotnet run in the terminal). This will execute the Program.cs entry point, which runs the Lab1.cs code to create the agent. The agent then uses an LLM to process the input prompt, generate a query, and connect to the database to fetch the results. each agent is created under each separate thread task.

Function calling enables Large Language Models to interact with external systems. The LLM determines when to invoke a function based on instructions, function definitions, and user prompts. The LLM then returns structured data that can be used by the agent app to invoke a function.

Dynamic SQL Generation

When the app starts, it incorporates the database schema and key data into the instructions for the Foundry Agent Service. Using this input, the LLM generates SQLite-compatible SQL queries to respond to user requests expressed in natural language.

From a security perspective, we must address the risk of SQL injection. Because the LLM generates queries dynamically, a malicious prompt could attempt to create a query that modifies or destroys data. Therefore, a critical best practice is to grant the database connection read-only permissions. This mitigates the risk by ensuring that even a successful injection attack cannot alter or delete data.

Lab 2 demonstrates how to query data using an LLM model and the file search tool. The file used for the search is uploaded at runtime

Grounding a conversation with documents is highly effective, especially for retrieving product details that may not be available in an operational database. The Foundry Agent Service includes a File Search tool, which enables agents to retrieve information directly from uploaded files, such as user-supplied documents or product data, enabling a RAG-style search experience.

When the app starts, a vector store is created, the sales datasheet PDF file is uploaded to the vector store, and it is made available to the agent.

Other tools like code interpreter and multilingual visualizations are present in Lab 3 and Lab 4, which can be referred at -

The sales agent is a hackathon project from the Microsoft agent hackathon tutorials.

I hope this post helped you understand how .NET developers can use agents in this space. If you found this article helpful, please consider following me for more content.

As a .NET developer, I am eager to explore the integration of artificial intelligence (AI) into my applications. Given the growing hype surrounding AI, I am particularly interested in understanding the practical steps involved in its adoption.

New AI models are constantly being released, and major tech companies are constantly competing to develop their own capabilities. This has prompted me to consider how I can leverage AI to enhance my applications.

I’ve decided to learn Artificial Intelligence (AI) as a C# developer to advance my professional development. Microsoft offers extensive documentation that serves as an entry point to connecting AI-related models and integrating them into applications. Consequently, I have commenced my learning process with ML.Net Model Builder, which serves as the foundational component for this integration.

General Explanation of the Machine Learning Concept

Machine learning is a branch of artificial intelligence (AI) that focuses on enabling computers to learn from data and improve their performance over time without explicit programming. In this context, the data is being trained using machine learning algorithms.

Requirements:

• Visual Studio Code

• Packages related to ML.Net

• SDK: dotnet 8 SDK , ML.Net CLI

In this demonstration, I will illustrate the process of constructing a machine learning model using the ML.Net CLI. Subsequently, I will demonstrate how to execute this command to generate a zip file containing the model. Finally, I will guide you on how to incorporate the generated model into a RESTful API. This example serves as a fundamental introduction to ML.Net for developers proficient in .NET programming.

Problem statement:
Construct a machine learning model capable of predicting the fare for a taxi ride, considering diverse input variables such as vendor identification, rate code, passenger count, trip duration in seconds, distance travelled, and payment method.

Background:

Taxi fare prediction is a pivotal component of the transportation sector, enabling both drivers and passengers to accurately estimate the cost of a ride. Precise fare predictions not only elevate customer satisfaction but also facilitate the optimisation of pricing strategies and contribute to the overall efficiency of taxi services.

Data Description:

The dataset contains historical data of taxi rides with the following features:

• Vendor ID: A unique identifier for the taxi vendor.

• Rate Code: The rate code indicating the fare structure.

• Passenger Count: The number of passengers in the taxi.

• Trip Time in Seconds: The duration of the trip in seconds.

• Trip Distance: The distance covered during the ride (in kilometers or miles).

• Payment Type: The method of payment used (e.g., cash, credit card).

• Fare Amount: The total fare charged for the ride (target variable).

Below is the project structure .

Step 1:

Create a New Solution for the Demo Application.

In the first step, create a new solution in Visual Studio Code and create a folder named “Data” to store the sample data file.

To initiate the model generation process, we require the data for prediction. I will be utilising the sample data provided in the Microsoft documentation: https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/taxi-fare-train.csv. Please prepare this CSV file and load it into the newly created “Data” folder as per above project structure .


Step 2:

To generate the console application for our Model project, execute the following command:

mlnet regression - dataset “Data/taxiFare.csv” - label-col 6 - has-header true - name RegressionModel - train-time 60

Various mlnet commands are the below in order to train and develop the custom models .

Subsequently, the RegressionModel project is created, comprising the following files:

- Program.cs: This serves as the entry point to the application, where input data is passed to the Model and its output is extracted.

- mbconfig: This configuration file pertains to our Model project, detailing the CPU utilised for Model execution, column properties, and other relevant information.

- evaluate.cs class: The evaluate.cs class facilitates the evaluation of the model’s performance. Model Builder utilises the trained model to generate predictions on new test data, subsequently assessing the quality of these predictions.

- training.cs: This file is employed for training the model based on the provided data. Training is an automated process that instructs the Model Builder on how to respond to queries within the specified scenario. Once trained, the model can make predictions on input data that it has not encountered previously. For instance, if the objective is to predict taxi prices based on the input columns, the trained model will be capable of generating predictions on unseen data.

When running the console application , basically we are evaluating the model based on the internal algorithm which are already defined for the ML.Net task — Regression , the model get trained and processed in to .mlnet zip file

When training a model, it is possible to observe the algorithms it employs and the accuracy it will achieve based on their scores. As a developer learning about ML.Net, there is no need to be concerned about the intricate deep algorithms used behind the machine learning scenarios. Our initial step should be to comprehend how the ML model is generated and integrated. Once we have gained confidence in our overall understanding, we can focus on the data training process and identify algorithms that can be modified in the existing generated code to train our model for optimal accuracy scores.

Here’s a screenshot of when the model is training. It’s exploring all the available algorithms.

Now your model is ready to use it .

Let’s run the console application in order to test the model , navigate to your Model project and run the below command:

dotnet run

Output:

Step 3 : Consume the Model through Rest api .

Create the dotnet web api project using below command:

dotnet new webapi -n TaxiFareAPi -o TaxiFareApi - no-https

Add the below nuget packages for the API project in the .csproj file:

<PackageReference Include=”Microsoft.AspNetCore.OpenApi” Version=”9.0.0” />
<PackageReference Include=”Microsoft.ML” Version=”4.0.2” />
<PackageReference Include=”Swashbuckle.AspNetCore” Version=”8.1.1” />
<PackageReference Include=”Microsoft.ML.FastTree” Version=”3.0.1” />

Below are the Controller, Service, and Input/Output classes. The model consumption is implemented in the service class using the ML context and is subsequently called by the controller. This process ensures that the model is consumed.

Upon constructing the API project and executing it on localhost, In order to test this on Postman will yield the following output:

Finally, we are able to comprehend the ML.NET Module on the dotnet platform. It offers a wide range of machine learning models for training and evaluating data to achieve greater accuracy, which can then be integrated into applications based on the problem statement. The ML.NET documentation provides various ML tasks and underlying algorithms for exploration. Additionally, Azure Open AI services offer pre-built models that can be easily integrated into our applications.

We can conclude that AI has the potential to perform more complex tasks and do so at an unprecedented speed. However, it is crucial to exercise caution when handling data, ensuring that it is properly prepared, trained, and evaluated. By adopting a systematic approach to organising all modules, we can harness the full potential of AI in our applications.

Next Steps in Learning in This Space:

- Practice various problem statements.

- Train models effectively with high accuracy.

- Understand the AI workflow, RAG, AI agents, and the latest buzzword, MCP (Model Context Protocol).

References:

https://learn.microsoft.com/en-us/dotnet/machine-learning/overview

In the world of .NET Core, Redis plays a crucial role in enhancing the performance of an application by reducing the load on the database. In this comprehensive blog post, we will delve deep into the implementation of Redis Cache in Dot Net Core, unraveling its mysteries and exploring its pivotal role in system design.

Caching is a technique that stores frequently accessed data in memory so that it can be retrieved quickly without having to access the underlying database. This can improve the performance and scalability of web applications by reducing the load on the database.

Redis Cache, an open-source, in-memory data structure store, acts as a lightning-fast cache. Implemented seamlessly in Dot Net Core, it revolutionizes how we handle data, ensuring rapid access and retrieval, even for the most complex applications.

Why Redis Cache?

Before we dive into the implementation details, let’s understand why we need Redis Cache. In any application, data retrieval from a database is a time-consuming process. To improve this, we use caching. Caching refers to the process of storing frequently accessed data in a temporary storage area (cache). So, when the same data is requested again, it is retrieved from the cache instead of the database. This significantly improves the performance and speed of data retrieval.

Redis Cache comes into play here. It stores the frequently accessed data in-memory and retrieves it at a much faster rate compared to traditional databases

Redis isn’t just a cache; it’s a versatile data store with various types catering to diverse data needs. From simple strings to intricate sorted sets, grasp the nuances of Redis data types. Understand when to use hashes for efficient storage, or when to opt for sorted sets to handle ranking and scoring seamlessly. Uncover real-world scenarios where each data type shines, empowering you to make informed design decisions in your Dot Net Core projects.

Redis Data Types

Redis supports various types of data structures:

• Strings: The most basic Redis data type that represents a sequence of bytes.

• Lists: Lists of strings sorted by insertion order.

• Sets: Unordered collections of unique strings.

• Hashes: Collections of field-value pairs.

• Sorted Sets: Collections of unique strings that maintain order by each string’s associated score.

• Bitmaps: Allow you to perform bitwise operations on strings.

• HyperLogLogs: Provide probabilistic estimates of the cardinality (i.e., number of elements) of large sets.

• Geospatial Indexes: Useful for finding locations within a given geographic radius or bounding box.

• Streams: Act like an append-only log that helps record events in the order they occur and then syndicate them for processing.

Implementing Redis Cache in Dot Net Core

1. First, we will install and configure Redis — to do this, grab the Redis image from Docker, as seen below:

command to get the redis image: docker pull redis

once it is done, run the Redis image:

• d: Run in the background

• -p-: Set the Port number

• — name: name of the image

docker -d -p 6279:6279 --name dotnet-redis redis

Once the Redis image setup is done successfully, verify whether the docker image is running or not by using the command: docker ps

2. Next, Create a new .Net core API project and install the Nuget package named: Microsoft.Extensions.Caching.StackExchangeRedis

3. We follow the repository pattern here to go through the Redis implementation.

Assume that we have a repository class called Basket. The function of this Basket class is to retrieve and update the Basket details, such as items that are moved to the Basket and updated there. To expedite this process and avoid repeatedly retrieving the data from the Database, we will store the Basket class operations data in the Redis cache.

//interface 

using System;
using Basket.API.Entities;

namespace Basket.API.Repositories
{
    public interface IBasketRepository
    {
        Task<ShoppingCart> GetBasket(string userName);

        Task<ShoppingCart> UpdateBasket(ShoppingCart basket);

        Task DeleteBasket(string userName);
    }
}


//BasketRepository class

using System;
using Basket.API.Entities;
using Microsoft.Extensions.Caching.Distributed;
using Newtonsoft.Json;

namespace Basket.API.Repositories
{
    public class BasketRepository : IBasketRepository
    {
        private readonly IDistributedCache _redisCache;

        public BasketRepository(IDistributedCache redisCache)
        {
            _redisCache = redisCache ?? throw new ArgumentNullException(nameof(redisCache));
        }

        public async Task<ShoppingCart> GetBasket(string userName)
        {
            var basket = await _redisCache.GetStringAsync(userName);

            if (String.IsNullOrEmpty(basket))
                return null;

            return JsonConvert.DeserializeObject<ShoppingCart>(basket);
        }

        public async Task<ShoppingCart> UpdateBasket(ShoppingCart basket)
        {
            await _redisCache.SetStringAsync(basket.UserName, JsonConvert.SerializeObject(basket));

            return await GetBasket(basket.UserName);
        }

        public async Task DeleteBasket(string userName)
        {
            await _redisCache.RemoveAsync(userName);
        }
    }
}

The Redis cache is injected into this class and inside the method we are setting and getting the items as per the username.

Configure the Redis server in the appsettings.json file :

 "CacheSettings": {
    "ConnectionString": "localhost:6379"
  },

Setting up the dependency injection for Basket service and Redis cache in program.cs file:

builder.Services.AddScoped<IBasketRepository, BasketRepository>();

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration.GetValue<string>("CacheSettings:ConnectionString");
});

The basic implementation of Redis in the dot net core project is seen above; once the Redis consumed methods are completed, these are called from the controller action methods.

Then I installed the RedisInsights client tool which is connected to redis server running on port : 6379 to view the data: you can see we used the Hash datatype to store the data in a key-value format:

Advantages of using Redis Cache in Dot Net Core

Redis Cache isn’t just a tool; it’s a solution. Explore the myriad advantages it offers, from unparalleled speed and scalability to fault tolerance and seamless integration with Dot Net Core technologies. Learn how Redis Cache enhances your project’s resilience, enabling it to handle surges in user traffic effortlessly. Understand its role in enhancing overall user satisfaction, making it a must-have component in your development toolkit.

• Improved performance: Redis Cache can significantly improve the performance of web applications by reducing the load on the database.

• Scalability: Redis Cache is a distributed cache, which means that it can be scaled horizontally to meet the needs of even the most demanding applications.

• Reliability: Redis Cache is a reliable and durable cache, with features such as replication and persistence to prevent data loss.

• Easy to use: Redis Cache is easy to use and provides a simple API for storing and retrieving data from the cache.

• Data Structures: Unlike other key-value stores that offer string-to-string mappings, Redis offers numerous data structures.

When and how to use Redis Cache in Dot Net Core

Redis Cache can be used in a variety of scenarios in Dot Net Core applications, including:

• Caching frequently accessed data: Redis Cache can be used to cache frequently accessed data, such as product data, user data, and configuration data.

• Output caching: Redis Cache can be used to cache the output of controllers and services, which can improve the performance of dynamic pages.

• Session caching: Redis Cache can be used to cache session data, which can improve the performance of web applications and reduce the load on the database.

• Message queuing: Redis Cache can be used as a message queue to implement asynchronous communication between different parts of an application.

Redis Cache can play an important role in system design by improving the performance, scalability, and reliability of web applications.

In this blog post, we’ve unraveled the mysteries of Redis Cache in the Dot Net Core framework and understood its role in system design. We also walked through a step-by-step process of implementing Redis Cache using StackExchange.Redis and configuring it in a .NET Core application. Finally, we highlighted some advantages of using Redis Cache such as high performance, scalability, persistence, and support for various data structures. you’re now equipped with the knowledge to transform your projects. Redis Cache isn’t just a tool; it’s your ticket to creating blazing-fast, reliable, and efficient applications.

Redis Cache in Dot Net Core isn’t merely a technological advancement; it’s a paradigm shift. Embrace it, integrate it, and witness the transformation it brings to your projects. Your users deserve nothing less than exceptional performance, and Redis Cache ensures you deliver exactly that.

Happy Coding!

Embarking on a journey into the realm of design patterns, let’s delve into the intricacies of the CQRS (Command Query Responsibility Segregation) pattern and its implementation in the .NET Core framework.

Understanding CQRS: A Paradigm Shift in Design

CQRS, an architectural pattern, introduces a fundamental shift in the way we design and structure our applications. By segregating the responsibilities of handling commands (write operations) and queries (read operations), CQRS opens the door to enhanced scalability, maintainability, and flexibility. We’ll explore the anatomy of the pattern, its key components, and how it decouples the command and query pathways.

This separation allows you to independently scale and optimize your read and write workloads and offers a high level of flexibility.

The CQRS pattern is based on the principle that the data model used to update an application can be different from the model used to read data from it. This separation provides several benefits:

1. Performance Optimization: By separating reads and writes, you can optimize each operation independently.

2. Complexity Management: CQRS allows you to handle complex business logic more effectively by focusing on one task at a time.

3. Scalability: With CQRS, you can scale your read and write operations independently, which is particularly useful in cloud-based applications.

However, implementing CQRS can also introduce some challenges:

1. Increased Complexity: CQRS can add complexity to your application, especially if your read and write operations are not significantly different.

2. Data Consistency: Since reads and writes are handled separately, there can be a delay in reflecting the updates in the read model.

Implementing CQRS in .NET Core: Bringing Theory to Reality

With the powerful ecosystem of .NET Core, implementing CQRS becomes an exciting endeavor. We’ll walk through practical examples, showcasing how to structure your application using separate models for commands and queries. Learn about the role of command handlers, query handlers, and the intricate dance between them. Witness the elegance of CQRS in action as we build a sample application step by step.

Here’s a step-by-step guide on how to implement the CQRS pattern in a .NET Core application:

1. Define Your Commands and Queries: Commands are methods that change the state of your data, while queries are methods that read your data.

2. Create Command and Query Handlers: These handlers contain the logic for handling commands and queries.

3. Use MediatR Library: MediatR is a popular library in .NET Core for implementing the CQRS pattern. It provides a simple, unambitious mediator implementation in .NET.

4. Implement Command and Query Handlers: With MediatR, each command or query has its own handler. This separation makes it easier to maintain and test your application.

I have created a Microservice application in .Net core (using .Net 6) which follows the Clean Architecture. Below is the folder structure .

The Ordering API service (/checkoutorder) is responsible for checking out the order request.

Under the CQRS pattern notion, queries and commands are developed individually; if we need to acquire data from the database, we use Queries; if we need to change or enter data into the database, we use Commands.

Queries :

The Handler class is found in the GetOrderList subdirectory. When we invoke it through the mediator, it connects to the DB repository interface to retrieve the OrderList.

//class : GetOrdersListQuery

using System;
using MediatR;

namespace Ordering.Application.Features.Orders.Queries.GetOrdersList
{
    public class GetOrdersListQuery : IRequest<List<OrdersVm>>
    {
        public string UserName { get; set; }

        public GetOrdersListQuery(string userName)
        {
            UserName = userName ?? throw new ArgumentNullException(nameof(userName));
        }
    }
}




//class : GetOrdersListQueryHandler

using System;
using AutoMapper;
using MediatR;
using Ordering.Application.Contracts.Persistence;

namespace Ordering.Application.Features.Orders.Queries.GetOrdersList
{
    public class GetOrdersListQueryHandler : IRequestHandler<GetOrdersListQuery, List<OrdersVm>>
    {
        private readonly IOrderRepository _orderRepository;
        private readonly IMapper _mapper;

        public GetOrdersListQueryHandler(IOrderRepository orderRepository, IMapper mapper)
        {
            _orderRepository = orderRepository ?? throw new ArgumentNullException(nameof(orderRepository));
            _mapper = mapper ?? throw new ArgumentNullException(nameof(mapper));
        }

        public async Task<List<OrdersVm>> Handle(GetOrdersListQuery request,
            CancellationToken cancellationToken)
        {
            var orderList = await _orderRepository.GetOrdersByUserName(request.UserName);
            return _mapper.Map<List<OrdersVm>>(orderList);
        }
    }
}

Commands:

For the checkout order command, we are inserting the new order entity into the database. Before inserting this request into the database, we can validate the parameters of the request using Fluent validation, as shown below. If the validation is successful, the handler will proceed with the insertion using the AddAsync() method.

using System;
using FluentValidation;

namespace Ordering.Application.Features.Orders.Commands.CheckoutOrder
{
    public class CheckoutOrderCommandValidator : AbstractValidator<CheckoutOrderCommand>
    {
        public CheckoutOrderCommandValidator()
        {
            RuleFor(p => p.UserName)
                .NotEmpty().WithMessage("{UserName} is required.")
                .NotNull()
                .MaximumLength(50).WithMessage("{UserName} must not exceed 50 characters.");

            RuleFor(p => p.EmailAddress)
               .NotEmpty().WithMessage("{EmailAddress} is required.");

            RuleFor(p => p.TotalPrice)
                .NotEmpty().WithMessage("{TotalPrice} is required.")
                .GreaterThan(0).WithMessage("{TotalPrice} should be greater than zero.");
        }
    }
}


// Hanlder class  :

using System;
using AutoMapper;
using MediatR;
using Microsoft.Extensions.Logging;
using Ordering.Application.Contracts.Infrastructure;
using Ordering.Application.Contracts.Persistence;
using Ordering.Application.Models;
using Ordering.Domain.Entities;

namespace Ordering.Application.Features.Orders.Commands.CheckoutOrder
{
    public class CheckoutOrderCommandHandler : IRequestHandler<CheckoutOrderCommand, int>
    {
        private readonly IOrderRepository _orderRepository;
        private readonly IMapper _mapper;
        private readonly IEmailService _emailService;
        private readonly ILogger<CheckoutOrderCommandHandler> _logger;

        public CheckoutOrderCommandHandler(IOrderRepository orderRepository, IMapper mapper, IEmailService emailService, ILogger<CheckoutOrderCommandHandler> logger)
        {
            _orderRepository = orderRepository ?? throw new ArgumentNullException(nameof(orderRepository));
            _mapper = mapper ?? throw new ArgumentNullException(nameof(mapper));
           _logger = logger ?? throw new ArgumentNullException(nameof(logger));
        }

        public async Task<int> Handle(CheckoutOrderCommand request, CancellationToken cancellationToken)
        {
            var orderEntity = _mapper.Map<Order>(request);
            var newOrder = await _orderRepository.AddAsync(orderEntity);

            _logger.LogInformation($"Order {newOrder.Id} is successfully created.");

            await SendMail(newOrder);

            return newOrder.Id;
        }

        
    }
}

Below is the code snippet to call this handler from the controller.

// To get list 
[HttpGet("{userName}", Name = "GetOrder")]
        [ProducesResponseType(typeof(IEnumerable<OrdersVm>), (int)HttpStatusCode.OK)]
        public async Task<ActionResult<IEnumerable<OrdersVm>>> GetOrdersByUserName(string userName)
        {
            var query = new GetOrdersListQuery(userName);
            var orders = await _mediator.Send(query);
            return Ok(orders);
        }

        // to insert into the DB 
        [HttpPost(Name = "CheckoutOrder")]
        [ProducesResponseType((int)HttpStatusCode.OK)]
        public async Task<ActionResult<int>> CheckoutOrder([FromBody] CheckoutOrderCommand command)
        {
            var result = await _mediator.Send(command);
            return Ok(result);
        }

In the realm of modern application development, CQRS stands as a beacon of innovation. “Mastering CQRS Pattern in .NET Core” empowers you to harness its potential, providing a comprehensive guide to implementing, understanding, and mastering this transformative pattern.

In the dynamic world of modern software development, serverless architectures are revolutionizing the way we build scalable and cost-effective applications. AWS Lambda, Amazon’s serverless computing service, empowers you to run code without the complexities of managing servers. When combined with Amazon API Gateway, you can create robust APIs to serve as front doors to your serverless logic.

If you’re a.NET developer, this blog post will help you use the potential of AWS Lambda and API Gateway. We’ll look at what these services are, why they matter, and walk through a practical example to help you understand.

This post provides a quick introduction to AWS Lambda serverless technology for experienced developers with a background in.Net. It explains how it is configured and how to integrate it with practical examples.

What is AWS Lambda?

Let’s break it down:

• Functions as a Service (FaaS): AWS Lambda adheres to the FaaS model. You write your core logic as self-contained functions. AWS handles the heavy lifting — provisioning, scaling, and managing the underlying infrastructure.

• Event-Driven: Lambda functions don’t run continuously. They are invoked in response to specific events/triggers like an HTTP request, a file upload to an S3 bucket, or a database record change.

• Pay-Per-Execution: You pay only for the compute time your functions consume. This means no costs are incurred when your code isn’t running.

Now let's see when to use the AWS Lambda?

• Event-Driven Workloads: If your application logic reacts to specific events (file uploads, API requests, database changes), Lambda’s event-driven nature is ideal.

• Short-Lived Tasks: Lambda excels at handling quick, stateless tasks. Don’t use it for long-running processes or applications requiring persistent connections.

• Cost-effectiveness: For applications with unpredictable or fluctuating workloads, the pay-per-execution model keeps costs low.

• Focus on Code: Want to ditch server management and focus on core logic? Lambda lets you do just that.

• Microservices Architecture: Break down monolithic applications into smaller, scalable Lambda functions for improved maintainability and deployment flexibility.

Not to use AWS Lambda:
Long-Running Processes: Lambda functions have a timeout limit .If your tasks or code execution time is more don’t use the lamda because the close is more for maintaining it hence other alternative are the containerized services like Amazon EC2 or Amazon ECS.

State Management: Lambda functions are inherently stateless. If your application requires complex state management, consider managed services like Amazon DynamoDB or ElastiCache.

What is Amazon API Gateway?

Generally An API Gateway, acts as a central hub for managing APIs (Application Programming Interfaces). It sits between client applications and backend services, providing a unified interface for interacting with various functionalities offered by the backend. Here’s a breakdown of its key functionalities:

• Single Entry Point: API Gateway serves as a single endpoint for all API requests, simplifying client interactions. Imagine it as a single front door leading to all the functionalities within a building.

• Routing: It efficiently routes incoming API requests to the appropriate backend services based on predefined rules. This routing logic can be based on factors like request path, headers, or parameters.

• Security: API Gateway enforces security measures like authentication and authorization. It ensures only authorized users or applications can access specific functionalities.

• Throttling and Rate Limiting: It safeguards your backend services from overload by throttling or rate-limiting requests. This prevents denial-of-service attacks and ensures smooth performance.

• Monitoring and Analytics: API Gateway provides valuable insights into API usage patterns. You can monitor metrics like latency, traffic volume, and error rates to identify bottlenecks and optimize performance.

• API Versioning: It facilitates managing different API versions. You can control access to specific versions and seamlessly roll out updates without disrupting existing client applications.

• Transformations: API Gateway can manipulate request and response data formats (e.g., JSON to XML) to ensure compatibility between different systems.

API gateway on Amazon AWS:

• AWS-Specific Integrations: It seamlessly integrates with other AWS services like Lambda, DynamoDB, and S3, making it easy to build serverless applications.

• Flexible API Design: Supports both RESTful APIs and WebSockets, catering to diverse application needs.

• Pay-Per-Use Model: You only pay for the resources consumed by your APIs, making it cost-effective for varying traffic loads.

• Developer Friendly: AWS offers robust tools and SDKs for various programming languages, making the development and management of APIs on API Gateway straightforward.

Why Lambda and API Gateway for .NET Developers?

Here’s where the magic happens for .NET developers:

1. Focus on Code: Your primary concern becomes your business logic, not infrastructure hassles. This leads to shorter development cycles and faster iterations.

2. Infinite Scalability: Lambda functions scale automatically to meet traffic demands. Whether it’s ten requests or a thousand, AWS handles it seamlessly.

3. Cost-Efficiency: The pay-per-execution model means you don’t pay for idle resources. This is especially beneficial for unpredictable or fluctuating workloads.

4. .NET Excellence: AWS provides excellent support for .NET on Lambda. You can use your favorite languages (C#, F#, etc.) and leverage familiar tools and libraries.

5. Ecosystem Integration: Lambda functions effortlessly connect with other AWS services like S3, DynamoDB, SNS, and more, creating powerful event-driven architectures.

Now hands on code — lets practically implement the Lambda function.

Prerequisite software, access, and tools must be installed on your machine.

1. Visual Studio 2022 (extensions of AWS)

2. Dotnet SDK (https://dotnet.microsoft.com/en-us/download)

3. AWS account with IAM-user created

4. AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

Create the user on AWS in order to access the resources :

Under the Users section create a new User :

attach the administrator access click next and then create a user.

Create the access key for this newly created user, which is used for connection to an AWS account from the AWS CLI .

select the Command Line Interface in next window and then give the description tag .

After creating the Access key, save it somewhere safe or download the.csv file at the bottom. Remember that you will need this Access key and Id to access to your AWS account via CLI, as mentioned below.

To verify the AWS CLI :

Now connect to your AWS account using AWS CLI tool:

aws configure --profile profileName 
aws configure list-profiles  // to verify the profile crated 

After connecting, validate the profile and install AWS templates and lambda tools using the dotnet command (requires dotnetSDK). The following are the commands.

dotnet tool install -g Amazon.Lambda.Tools
dotnet new install Amazon.Lambda.Templates

Now create the function and deploy to AWS using below commands :

dotnet new lambda.EmptyFunction --name functionName
dotnet lambda function-deploy  functionName

While deploying the function to AWS , it will ask for roles, so create one and give access to newly created roles: This role is needed because the function needs access to connect to logs or any other resources like dynamodb.

Once the function deployment is successful , you can see the newly created function under the Lamda AWS , like below :

In the newly Created function : aws-lambda-tools-defaults.json is important where all the necessary details of the function are configured, entry point of the function is :FunctionHandler.

Run the function app from the AWS console, like below : Give the input string, which is converted to uppercase.

To check the logs go to CloudWatch like shown in above image.

In this blog post, we explored how to implement and understand AWS Lambda using .NET as the backend language. We discussed the benefits of using Lambda for serverless computing and provided a step-by-step guide to creating and deploying a Lambda function.

By following these steps,.NET developers can leverage the power of AWS Lambda to build scalable, cost-effective, and flexible applications in the cloud. Whether you’re building APIs, processing data, or handling backend logic, AWS Lambda with .NET opens up a world of possibilities for your next project.

So, if you’re ready to dive into serverless computing and explore the endless opportunities it offers, start experimenting with AWS Lambda and .NET today!

In a future post, I’ll try to describe how the API Gateway authenticates to the Lambda function; keep tuned.

Happy Learning.

Read more

The .NET Core framework provides powerful tools and mechanisms to handle various configuration settings, making it easier for developers to securely store sensitive and non-sensitive information. In this blog post, we will delve into the concept of configuration settings in .NET Core, explore the mechanisms and patterns used for this purpose, and guide you on the path to becoming a configuration settings maestro.

Before we dive into the intricacies of configuration settings, let’s understand why they are essential. Configuration settings are the parameters and values that dictate how your application behaves. They encompass a wide range of information, from database connection strings and API keys to application-specific settings like the number of threads for parallel processing or feature flags. The ability to adjust these settings without changing the code allows for flexibility, scalability, and enhanced security.

In a .NET Core application, configuration settings can be used to control application behavior, manage external service integrations, and store sensitive data such as connection strings, secrets, and authentication tokens. But how do you effectively manage and access these settings? That’s where the magic of .NET Core’s configuration system comes into play.

ways to get the Key-Value pairs in the c# code:

Using the app.configuration() method we can get the key values in the program.cs file like

app.UseEndpoints(endpoints => {

  endpoints.Map("/", async context =>
  {
  
  await context.Response.WriteAsync(app.Configuration.GetValue<string>("Key") + "\n");

});

});

By using Iconfiguration inside the controller :

public class HomeController : Controller
  {
    //private field
    private readonly IConfiguration _configuration;

    //constructor
    public HomeController(IConfiguration configuration)
    {
      _configuration = configuration;
    }

    [Route("/")]
    public IActionResult Index()
    {
//Hierarchical Configuration
//var getConnection= _configuration.GetSection("DBConnection");
//getConnection["ConnectionString"]
      ViewBag.MyKey = _configuration["Key"];
      ViewBag.MyAPIKey = _configuration.GetValue("APIKey", "the default key");

      return View();
    }
  }

Using the options pattern, we can tie the JSON setting to the class model: Assume we have an API client ID and client secret in the appsettings.These values are bound to a class in the JSON file and then used as follows:

Options pattern (strongly typed)uses custom classes to specify what configuration settings are to be loaded into properties.

//appsetting.json
{
"apiValues":
{
  "ClientID": "",
  "ClientSecret":""
}
}



public class ApiOptions
  {
    public string? ClientID { get; set; }
    public string? ClientSecret { get; set; }
  }

// inside the action method 
 [Route("/")]
    public IActionResult Index()
    {
      
      ApiOptions options = new ApiOptions();
      _configuration.GetSection("apiValues").Bind(options);

      ViewBag.ClientID = options.ClientID;
      ViewBag.ClientSecret = options.ClientSecret;

      return View();
    }

Configuration as a service — instead of getting the values every time in each controller we can use the configuration as a service which is injected into any controller to get the key value fields :

//add this in program.cs file 
builder.Services.Configure<ApiOptions>(builder.Configuration.GetSection("apiValues"));

//in controller class
 public class HomeController : Controller
  {
    //private field
    private readonly ApiOptions _options;

    //constructor
    public HomeController(IOptions<ApiOptions> apiOptions)
    {
      _options = apiOptions.Value;
    }

    [Route("/")]
    public IActionResult Index()
    {
      ViewBag.ClientID = _options.ClientID;
      ViewBag.ClientSecret = _options.ClientSecret;

      return View();
    }
  }

Environment Specific Configuration

Order of Precedence of Configuration Sources

First, the key-value pairs in the appsettings.json are checked, If they are found, they are returned; if they are not found, they are searched in the following section, appsettingsEnvironment.json file. This environment will be located in the launchsettings.json file under the properties folder, the Precedence will function similarly to the values found in the most recent settings they are only fetched.

User Secrets

User secrets are a convenient way to store development-specific settings that should not be included in version control. You can use the dotnet user-secrets command to manage these secrets at the user level. For example, you can store a database connection string as a user secret:

If you want to store the keys that are outside the source control the user-secrets is the best place to configure it, the user-secrets are stored on the developer machine, and the secret key is created for this configuration.

Run the command: dotnet user-secret init

you can see the <UserSecretId> per developer machine is created, in order to add the new keys inside this secrets.json :

cmd: dotnet user-secrets set “FirstKey” “FirstValue”

To view the newly added secret keys right-click on the project and goto Manage secrets then the secrets.json file will be opened :

Environment Variables

Environment variables are an excellent choice for storing sensitive information because they can be securely managed at the system level. .NET Core allows you to load environment variables directly into your configuration: In the prod environment we don’t get any visual studio there to configure the values only the Terminal will be available, we can set configuration values as in-process environment variables.

$Env:ParentKey__ChildKey="value"
dotnet run --no-launch-profile

//__ (underscore and underscore) is the separator between parent key and child key.

Custom Json Configuration

We can also add custom JSON files to contain the key-value pairs; this was utilized in large-scale projects where a lot of configuration needed to be preserved; in this idea, the custom JSON file is bonded to the model class.

builder.Host.ConfigureAppConfiguration( (hostingContext, config) => {
    config.AddJsonFile("filename.json", optional: true, reloadOnChange: true);
});

Azure Key Vault

When working with sensitive data in a secure way, Azure Key Vault is a popular choice. .NET Core provides an Azure Key Vault configuration provider, allowing you to fetch secrets and configuration settings from Azure Key Vault securely. This is ideal for managing production secrets and certificates.

var azureKeyVaultEndpoint = "https://your-key-vault.vault.azure.net/";
var configuration = new ConfigurationBuilder()
    .AddAzureKeyVault(azureKeyVaultEndpoint)
    .Build();

Configuring your .NET Core application effectively is a crucial aspect of software development. With the powerful configuration system provided by .NET Core, you can easily manage settings, both sensitive and non-sensitive, to build robust and flexible applications. By following best practices and patterns, such as strongly typed configuration and secrets management, you can ensure your application is secure and scalable.

In a world where data security is paramount, .NET Core’s configuration management capabilities empower developers to navigate the complexities of sensitive information confidently. Whether you’re safeguarding API keys, managing database connections, or fine-tuning application behavior, a deep understanding of .NET Core’s configuration settings will undoubtedly set you on the path to success.

So, the next time you embark on a.NET Core project, remember that configuration settings are more than just variables; they are the cornerstone that assures your application’s integrity and resilience in the face of changing problems. Accept the power of configuration management and watch your .NET Core projects fly to new heights of efficiency and security. Happy coding!

🌐 Connect with Me:

🔗 Medium: https://medium.com/@dileepsreepathi
🔗 LinkedIn: https://www.linkedin.com/in/dileep-sreepathi/

Dileep’s Substack

I'm thrilled to have you here, and I'm excited to embark on this journey of exploration and learning together. In this space, I plan to share my thoughts, insights, and experiences on a wide range of topics in the field of Computer Science and Engineering

Read more

Data consistency is one of the most important considerations in the system design process for large-scale distributed systems. It ensures that all users see the same data at the same time, even as updates and modifications are being made.

In this blog post, we will delve into the complexities of data consistency, exploring its implementation, understanding its nuances, and discussing the various types of data consistency. By the end of this journey, you will not only comprehend the intricacies of this critical concept but also appreciate its role in enhancing the efficiency of accessing data from databases in large-scale distributed systems.

Understanding Data Consistency: A Foundation of System Design

At its core, data consistency refers to the uniformity of data across all nodes in a distributed system. In simpler terms, it ensures that when multiple users access the system simultaneously, they all see the same set of data. Achieving data consistency is challenging, especially in large-scale distributed systems where data is spread across multiple servers, often in different geographical locations.

There are two main types of data consistency: strong consistency and eventual consistency. Strong consistency guarantees that all replicas of data are always up-to-date. Eventual consistency allows for temporary inconsistencies but guarantees that all replicas will eventually converge to the same view.

Implementing data consistency involves striking a delicate balance between performance and reliability. Several strategies can be employed, each with its own set of advantages and drawbacks.

1. Strong Consistency: The Gold Standard

Strong consistency guarantees that once a piece of data is updated, all subsequent accesses to that data will return the updated value. While this approach ensures data accuracy, it can impact performance due to the need for synchronization among distributed nodes. However, for applications where data accuracy is non-negotiable, strong consistency remains the gold standard.

• Pros: Ensures data accuracy, and simplifies application logic.

• Cons: This may lead to latency issues, especially in geographically dispersed systems, and increased complexity in implementation.

2. Eventual Consistency: Embracing the Asynchronous Nature

Eventual consistency, on the other hand, acknowledges that in a distributed system, it takes time for all nodes to receive updates. With eventual consistency, all nodes will converge to the same state given enough time, even if some temporary inconsistencies are observed. This approach prioritizes system availability and partition tolerance, making it suitable for systems where real-time consistency is not critical.

• Pros: Enhances system availability, improved performance, easier to implement.

• Cons: Temporary inconsistencies can confuse users, and complex conflict resolution mechanisms are required.

The choice of data consistency model depends on the specific requirements of the system. For example, systems that require high availability may be willing to sacrifice strong consistency for better performance and scalability. On the other hand, systems that require high data integrity may need to use strong consistency, even if it comes at a cost in terms of performance.

Implementing Data Consistency

There are a number of different techniques that can be used to implement data consistency in distributed systems. Some of the most common techniques include:

• Distributed transactions: Distributed transactions allow multiple database operations to be executed as a single unit, ensuring that all or none of the operations succeed. This can be used to implement strong consistency.

• Replication: Replication involves creating multiple copies of data and storing them on different servers. This can be used to improve availability and performance, but it can also make it more difficult to maintain data consistency.

• Versioning: Versioning allows multiple versions of the same data to be stored at the same time. This can be used to implement eventual consistency.

• Conflict resolution: Conflict resolution algorithms are used to resolve conflicts between different versions of data. This is important for systems that use eventual consistency.

Choosing the Right Data Consistency Model

The choice of data consistency model depends on the specific requirements of the system. Some factors to consider include:

• Availability: How important is it for the system to be available even if there are inconsistencies in the data?

• Performance: How important is it for the system to be performant, even if it comes at the cost of some data consistency?

• Integrity: How important is it for the data to be always consistent?

Examples of Data Consistency in Practice

Here are some examples of how data consistency is used in practice:

• Financial systems: Financial systems require high data integrity. Therefore, they typically use strong consistency models, even if it comes at a cost in terms of performance.

• Social media systems: Social media systems require high availability and performance. Therefore, they may be willing to sacrifice strong consistency for better performance and scalability.

• E-commerce systems: E-commerce systems need to be both available and reliable. They typically use eventual consistency models, with conflict resolution algorithms to ensure that data is eventually consistent.

Which Data Consistency Type Should You Choose?

The choice of data consistency type depends on the specific requirements of the system. If data consistency is critical, then you may need to use strong consistency, even if it comes at a cost in terms of performance and scalability. However, if availability and performance are more important, then you may be able to get away with using eventual consistency.

How to Implement Data Consistency Efficiently

There are a number of things you can do to implement data consistency efficiently:

• Use the right data consistency model: Choose the data consistency model that is most appropriate for your system's requirements.

• Use the right data structures: Choose data structures that are designed for data consistency. For example, you can use a distributed hash table (DHT) to store replicated data in a consistent way.

• Use caching: Caching can be used to improve performance and reduce the number of database

In the intricate landscape of large-scale distributed systems, achieving data consistency is akin to mastering a delicate dance between accuracy and efficiency. As architects of these systems, it is imperative to choose the right consistency model that aligns with the application's requirements. Understanding the nuances of strong, eventual, and causal consistency empowers designers to make informed decisions, ensuring that the system operates seamlessly even in the face of network partitions and high user loads.

In closing, the journey toward data consistency in system design is not without its challenges, but it is a journey well worth undertaking. By embracing the complexities and nuances of various consistency models, architects and developers can pave the way for efficient data access, creating robust and reliable systems that stand the test of time. Remember, in the world of large-scale distributed systems, data consistency isn't just a goal; it's the foundation upon which innovation and reliability thrive.