Learn .NET - Messaging & Integration

#Introduction

Modern applications rarely stand alone. Episode 17 covers messaging and integration — how .NET services communicate, process background work, and stay consistent when split into many services.

You will learn hosted services for background work, integration with message brokers such as RabbitMQ, Kafka, and Azure Service Bus, gRPC and REST communication, and how to handle consistency in distributed systems — where a single database transaction is no longer enough.

#Background Services and Hosted Services

#Creating a Worker Service

A hosted service is a service that runs for the lifetime of the application inside the Generic Host. The worker template provides the skeleton:

dotnet new worker -n OrderWorker

dotnet new worker -n OrderWorker creates a project with a Worker that implements BackgroundService — the ExecuteAsync method runs as soon as the host starts, and is stopped cleanly when the host shuts down.

#Example BackgroundService

A worker class processes a queue in a loop:

public class OrderWorker : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            await Task.Delay(TimeSpan.FromSeconds(30), stoppingToken);
            Console.WriteLine("Cek pesanan baru");
        }
    }
}

ExecuteAsync keeps running until stoppingToken is cancelled when the host stops. This pattern is used for scheduled jobs, cache cleanup, and background integrations.

#Message Brokers

#The Message Broker Concept

A message broker mediates communication between services through queues or topics. Producers send messages without waiting for consumers; consumers process at their own pace. This decouples services in time.

• RabbitMQ: a classic broker with flexible routing, easy to operate.
• Kafka: a distributed log for streaming and large events.
• Azure Service Bus: a managed Azure service with enterprise features.

#Sending Messages with MassTransit

The MassTransit library unifies broker integration with a clean abstraction:

dotnet add package MassTransit
dotnet add package MassTransit.RabbitMQ

Connection configuration:

builder.Services.AddMassTransit(x =>
{
    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.Host("localhost", "/", h =>
        {
            h.Username("guest");
            h.Password("guest");
        });
    });
});

UsingRabbitMq connects the bus to the broker. Once registered, services can send messages via IPublishEndpoint or receive them through consumers. Switching brokers (for example to Azure Service Bus) only requires changing one line of configuration — the business code stays the same.

#Choosing a Broker

A guide:

• RabbitMQ: complex routing, medium load, easy to self-host.
• Kafka: very high event volumes, log replay, streaming.
• Azure Service Bus: Azure integration, topics and sessions, no operations.

#gRPC and REST Integration

#gRPC for Inter-Service Communication

gRPC provides high performance with a binary protocol for inter-service communication. Contract definitions are written in a .proto file:

syntax = "proto3";
 
service Katalog {
  rpc GetProduk(ProdukId) returns (Produk);
}
 
message ProdukId { int32 id = 1; }
message Produk { string nama = 1; double harga = 2; }

gRPC compiles the .proto file into strongly typed C# clients and servers. For latency-sensitive internal communication between services, gRPC outperforms REST thanks to binary serialization and HTTP/2.

#REST at the System Boundary

Use REST at open system boundaries for external clients — browsers, mobile, and partners — because of its readability and universal tooling support. The common pattern: REST at the edge, gRPC inside. Both coexist in ASP.NET Core without conflict.

#Distributed Transactions and Eventual Consistency

#Why a Single Transaction Fails Here

In microservices, an operation spans many databases — a single ACID transaction does not apply. The pragmatic solution is eventual consistency: the system reaches consistency after a period of time, driven by events.

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string Tipe { get; set; }
    public string Payload { get; set; }
    public bool Dikirim { get; set; }
}

The transactional outbox pattern stores events in the same table as the business transaction, then a worker sends them to the broker. This guarantees events are not lost: if delivery fails, the message stays in the outbox and is retried.

#Sagas for Compensation

For long flows across services, use a saga: every step has a compensating counterpart. If a step fails, the saga runs the compensation of the previous steps — for example, cancel the order if payment fails. Consistency is achieved eventually, not atomically.

#Messaging Practice Summary

• Use BackgroundService for scheduled background work.
• MassTransit abstracts RabbitMQ, Kafka, and Azure Service Bus.
• Use gRPC inside the system, REST at the external boundary.
• The transactional outbox prevents event loss.
• Sagas provide compensation for cross-service flows.
• Keep messages idempotent for safe retries.

#Closing

Key takeaways:

• Hosted services run background work for the lifetime of the application.
• Message brokers decouple services in time.
• RabbitMQ, Kafka, and Azure Service Bus are chosen by volume and need.
• gRPC delivers performance for internal communication; REST for external.
• Eventual consistency replaces cross-service transactions.
• The transactional outbox and sagas maintain consistency and compensation.

In the next episode 18 we will discuss modern tooling and build automation — the dotnet CLI, MSBuild, and SDK-style projects, continuous integration with GitHub Actions, source generators and Roslyn analyzers, and code formatting, linting, and reproducible builds.