Brokers are a liaison between business logic and the outside world. They are wrappers around external libraries, resources, services, or APIs to satisfy a local interface for the business to interact with them without having to be tightly coupled with any particular resources or external library implementation.
Brokers, in general, are meant to be disposable and replaceable - they are built with the understanding that technology evolves and changes all the time. Therefore, at some point in the lifecycle of a given application, it will be replaced with a recent technology that gets the job done faster.
However, brokers also ensure that your business is pluggable by abstracting away any specific external resource dependencies from what your software is trying to accomplish.
For instance, you have an API built to consume and serve data from a SQL server. At some point, you decided that a better, more economical option for your API is to rely on a NoSQL technology instead. Having a broker to remove the dependency on SQL will make it much easier to integrate with NoSql with the least time and cost humanly possible.
In any application, mobile, desktop, web, or simply an API, brokers usually reside at the "tail" of any app because they are the last point of contact between our custom code and the outside world.
Whether the outside world is simply local storage in memory or an independent system that resides behind an API, they all have to reside behind Brokers in any application.
In the following low-level architecture for a given API - Brokers reside between our business logic and the external resource:
There are a few simple rules that govern the implementation of any broker - these rules are:
Brokers must satisfy a local contract and implement a local interface to allow decoupling between their implementation and the services that consume them.
For instance, given that we have a local contract, IStorageBroker that requires an implementation for any given CRUD operation for a local model Student - the contract operation would be as follows:
public partial interface IStorageBroker
{
ValueTask<IQueryable<Student>> SelectAllStudentsAsync();
}An implementation for a storage broker would be as follows:
public partial class StorageBroker
{
public DbSet<Student> Students { get; set; }
public async ValueTask<IQueryable<Student>> SelectAllStudentsAsync() =>
await SelectAllAsync<Student>();
}A local contract implementation can be replaced at any point, from utilizing the Entity Framework as shown in the previous example to using a completely different technology like Dapper or an entirely different infrastructure like an Oracle or PostgreSQL database.
Brokers should not have any form of flow control, such as if statements, while loops, or switch cases. That's because flow-control code is considered business logic, and it fits better in the services layer, where business logic should reside, not the brokers.
For instance, a broker method that retrieves a list of students from a database would look something like this:
public async ValueTask<IQueryable<Student>> SelectAllStudentsAsync() =>
await SelectAllAsync<Student>();A simple function that calls the native EntityFramework DbSet<T> and returns a local model like Student.
Exception handling is a form of flow control. Brokers are not supposed to handle exceptions but rather let them propagate to the broker-neighboring services, where they can be properly mapped and localized.
Brokers are also required to handle their configurations - they may have a dependency injection from a configuration object to retrieve and set up the configurations for whichever external resource they are integrating.
For instance, connection strings in database communications are required to be retrieved and passed into the database client to establish a successful connection, as follows:
public partial class StorageBroker : EFxceptionsContext, IStorageBroker
{
private readonly IConfiguration configuration;
public StorageBroker(IConfiguration configuration)
{
this.configuration = configuration;
this.Database.Migrate();
}
...
...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
string connectionString = this.configuration.GetConnectionString("DefaultConnection");
optionsBuilder.UseSqlServer(connectionString);
}
}Brokers may construct an external model object based on primitive types passed by broker-neighboring services.
For instance, in an e-mail notification broker, input parameters for a .Send(...) function require the basic input parameters such as the subject, content, or address. Here's an example:
public async ValueTask SendMailAsync(List<string> recipients, string subject, string content)
{
Message message = BuildMessage(recipients, ccRecipients, subject, content);
await SendEmailMessageAsync(message);
}The primitive input parameters will ensure no strong dependencies between the broker-neighboring services and the external models. Even in situations where the broker is simply a point of integration between your application and an external RESTful API, it's very highly recommended that you build your native models to reflect the same JSON object sent or returned from the API instead of relying on NuGet libraries, DLLs or shared projects to achieve the same goal.
The contracts for brokers shall remain as generic as possible to indicate their overall functionality; for instance, we say IStorageBroker instead of ISqlStorageBroker to indicate a particular technology or infrastructure.
With a single storage broker, it might be more convenient to maintain the same name as the contract. However, in the case of concrete implementations of brokers, it all depends on how many brokers you have that provide similar functionality. In our case, we have a concrete implementation of IStorageBroker, so the name would be StorageBroker.
However, if your application supports multiple queues, storage, or e-mail service providers, you might need to start specifying the overall target of the component; for instance, an IQueueBroker would have multiple implementations such as NotificationQueueBroker and OrdersQueueBroker.
However, if the concrete implementations target the same model and business logic, a diversion to the technology might be more beneficial. In this case, for instance, IStorageBroker, two different concrete implementations could be SqlStorageBroker and MongoStorageBroker. This case is typical in situations where the intention is to reduce production infrastructure costs.
Brokers speak the language of the technologies they support.
For instance, in a storage broker, we say SelectById to match the SQL Select statement; in a queue broker, we say Enqueue to match the language.
If a broker supports an API endpoint, it shall follow the RESTFul semantics, such as POST, GET, or PUT. Here's an example:
public async ValueTask<Student> PostStudentAsync(Student student) =>
await this.PostAsync(RelativeUrl, student);Brokers cannot call other brokers because they are the first point of abstraction and require no additional abstractions or dependencies other than a configuration access model.
Brokers also can't have services as dependencies, as the flow in any given system shall come from the services to the brokers and not the other way around.
For instance, even when a microservice has to subscribe to a queue, brokers will pass forward a listener method to process incoming events but not call the services that provide the processing logic.
The general rule here is that only services can call brokers, while brokers can only call external native dependencies.
Brokers supporting multiple entities, such as Storage brokers, should leverage partial classes to break down the responsibilities per entity.
For instance, if we have a storage broker that provides all CRUD operations for both Student and Teacher models, then the organization of the files should be as follows:
- IStorageBroker.cs
- IStorageBroker.Students.cs
- IStorageBroker.Teachers.cs
- StorageBroker.cs
- StorageBroker.Students.cs
- StorageBroker.Teachers.cs
The primary purpose of this particular organization, leveraging partial classes, is to separate the concern for each entity to a finer level, which should make the maintainability of the software much higher.
Broker file and folder naming conventions strictly focus on the plurality of the entities they support and the singularity of the overall resource supported.
For instance, we say IStorageBroker.Students.cs. We also say IEmailBroker or IQueueBroker.Notifications.cs - singular for the resource and plural entities.
The same concept applies to the folders or namespaces containing these brokers.
For instance, we say:
namespace OtripleS.Web.Api.Brokers.Storages
{
...
}And we say:
namespace OtripleS.Web.Api.Brokers.Queues
{
...
}In most applications built today, some common Brokers are usually needed to get an enterprise application up and running - some of these are Storage, Time, APIs, Logging, and Queues.
Some brokers interact with existing system resources, such as time, to allow broker-neighboring services to treat time as a dependency and control how a particular service would behave based on the value of time at any point in the past, present, or future.
Entity brokers provide integration points with the system's external resources to fulfill business requirements.
For instance, entity brokers integrate with storage, providing capabilities to store or retrieve records from a database.
Entity brokers are also like queue brokers, providing a point of integration to push messages to a queue for other services to consume and process to fulfill their business logic.
Broker-neighboring services can only call entity brokers because they require a level of validation on the data they receive or provide before proceeding.
Support brokers are general-purpose brokers. They provide the functionality to support services, but they have no characteristics that make them different from any other system.
An excellent example of a support broker is the DateTimeBroker, a broker specifically designed to abstract away the business layer's strong dependency on the system date time.
Time brokers don't target any specific entity; they are almost the same across many systems.
Another example of support brokers is the LoggingBroker - they provide data to logging and monitoring systems to enable the system's engineers to visualize the overall flow of data across the system and be notified in case any issues occur.
Support Brokers may be called across the entire business layer: foundation, processing, orchestration, coordination, management, or aggregation services, unlike Entity Brokers. Logging brokers are required as a supporting component in the system to provide all the capabilities needed for services to log their errors, calculate a date, or perform any other supporting functionality.
You can find real-world examples of brokers in the OtripleS project here.
Here's a real-life implementation of an entire storage broker for all CRUD operations for Student entity:
In order to abstract at an operational level all example implementations in this section will utilize the ValueTask type to provide an asynchronization abstraction. This approach encourages a consistent handling of asynchronous operations across all method implementations, promoting a clean and efficient abstraction that aligns with Standard principles.
namespace OtripleS.Web.Api.Brokers.Storages
{
public partial interface IStorageBroker
{
}
}using System;
using System.Linq;
using System.Threading.Tasks;
using EFxceptions.Identity;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using OtripleS.Web.Api.Models.Users;
namespace OtripleS.Web.Api.Brokers.Storages
{
public partial class StorageBroker : EFxceptionsIdentityContext<User, Role, Guid>, IStorageBroker
{
private readonly IConfiguration configuration;
public StorageBroker(IConfiguration configuration)
{
this.configuration = configuration;
this.Database.Migrate();
}
private async ValueTask<T> InsertAsync<T>(T @object)
{
this.Entry(@object).State = EntityState.Added;
await this.SaveChangesAsync();
return @object;
}
private async ValueTask<IQueryable<T>> SelectAllAsync<T>() where T : class => this.Set<T>();
private async ValueTask<T> SelectAsync<T>(params object[] @objectIds) where T : class =>
await this.FindAsync<T>(objectIds);
private async ValueTask<T> UpdateAsync<T>(T @object)
{
this.Entry(@object).State = EntityState.Modified;
await this.SaveChangesAsync();
return @object;
}
private async ValueTask<T> DeleteAsync<T>(T @object)
{
this.Entry(@object).State = EntityState.Deleted;
await this.SaveChangesAsync();
return @object;
}
...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
string connectionString = this.configuration.GetConnectionString("DefaultConnection");
optionsBuilder.UseSqlServer(connectionString);
}
}
}using system;
using system.Linq;
using system.Threading.Tasks;
using OtripleS.Web.Api.Models.Students;
namespace OtripleS.Web.Api.Brokers.Storages
{
public partial interface IStorageBroker
{
ValueTask<Student> InsertStudentAsync(Student student);
ValueTask<IQueryable<Student>> SelectAllStudentsAsync();
ValueTask<Student> SelectStudentByIdAsync(Guid studentId);
ValueTask<Student> UpdateStudentAsync(Student student);
ValueTask<Student> DeleteStudentAsync(Student student);
}
}using System;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using OtripleS.Web.Api.Models.Students;
namespace OtripleS.Web.Api.Brokers.Storages
{
public partial class StorageBroker
{
public DbSet<Student> Students { get; set; }
public async ValueTask<Student> InsertStudentAsync(Student student) =>
await InsertAsync(student);
public async ValueTask<IQueryable<Student>> SelectAllStudentsAsync() =>
await SelectAllAsync<Student>();
public async ValueTask<Student> SelectStudentByIdAsync(Guid studentId) =>
await SelectAsync<Student>(studentId);
public async ValueTask<Student> UpdateStudentAsync(Student student) =>
await UpdateAsync(student);
public async ValueTask<Student> DeleteStudentAsync(Student student) =>
await DeleteAsync(student);
}
}public interface IDateTimeBroker
{
ValueTask<DateTimeOffset> GetCurrentDateTimeOffsetAsync();
}public class DateTimeBroker : IDateTimeBroker
{
public async ValueTask<DateTimeOffset> GetCurrentDateTimeOffsetAsync() =>
DateTimeOffset.UtcNow;
}public interface ILoggingBroker
{
ValueTask LogInformationAsync(string message);
ValueTask LogTraceAsync(string message);
ValueTask LogDebugAsync(string message);
ValueTask LogWarningAsync(string message);
ValueTask LogErrorAsync(Exception exception);
ValueTask LogCriticalAsync(Exception exception);
}public class LoggingBroker : ILoggingBroker
{
private readonly ILogger<LoggingBroker> logger;
public LoggingBroker(ILogger<LoggingBroker> logger) =>
this.logger = logger;
public async ValueTask LogInformationAsync(string message) =>
this.logger.LogInformation(message);
public async ValueTask LogTraceAsync(string message) =>
this.logger.LogTrace(message);
public async ValueTask LogDebugAsync(string message) =>
this.logger.LogDebug(message);
public async ValueTask LogWarningAsync(string message) =>
this.logger.LogWarning(message);
public async ValueTask LogErrorAsync(Exception exception) =>
this.logger.LogError(exception.Message, exception);
public async ValueTask LogCriticalAsync(Exception exception) =>
this.logger.LogCritical(exception, exception.Message);
}Brokers are the first layer of abstraction between your business logic and the outside world. But they are not the only layer of abstraction because a few native models will still leak through your brokers to your broker-neighboring services. It is natural to avoid doing any mappings outside the realm of logic, in our case, the foundation services.
For instance, in a storage broker, regardless of the ORM used, some native exceptions from your ORM (EntityFramework, for example) will occur, such as DbUpdateException or SqlException. In that case, we need another layer of abstraction to act as a mapper between these exceptions and our core logic to convert them into local exception models.
This responsibility lies in the hands of the broker-neighboring services. I also call them foundation services; these services are the last point of abstraction before your core logic, which consists of local models and contracts.
Over time, some common questions arose from the engineers I worked with throughout my career. Since some of these questions reoccurred on several occasions, it might be helpful to aggregate them here so everyone can learn about other perspectives on brokers.
From an operational standpoint, brokers seem to be more generic than repositories.
Repositories usually target storage-like operations, mainly towards databases. However, brokers can be an integration point for any external dependency, such as e-mail services, queues, and other APIs.
A more similar pattern for brokers is the Unit of Work pattern. It mainly focuses on the overall operation without having to tie the definition or the name with any particular operation.
In general, all these patterns try to implement the same SOLID principles: separation of concern, dependency injection, and single responsibility.
But because SOLID are principles and not exact guidelines, it's expected to see all different implementations and patterns to achieve that principle.
1.7.1 Why can't the brokers implement a contract for methods that return an interface instead of a concrete model?
That would be an ideal situation, but that would also require brokers to do a conversion or mapping between the native models returned from the external resource SDKs or APIs and the internal model that adheres to the local contract.
Doing that on the broker level will require introducing business logic into that realm, which is outside the purpose of that component.
We define business logic code as any intended sequential, selective, or iteration code. Brokers are not unit-tested because they lack business logic. They may be part of an acceptance or integration test but certainly not part of unit-level tests simply because they do not contain business logic.
1.7.2 If brokers were a layer of abstraction from the business logic, why do we allow external exceptions to leak through them onto the services layer?
Brokers are only the first layer of abstraction, but not the only one. Broker-neighboring services are responsible for converting the native exceptions occurring in a broker into a more local exception model that can be handled and processed internally within the business logic realm.
Business logic emerges in the processing, orchestration, coordination, and aggregation layers, where all the exceptions, returned models, and operations are local to the system.
Since brokers must own their configurations, it makes more sense to partialize when possible to avoid reconfiguring every storage broker for each entity.
Partial classes are a feature in the C# language, but it should be possible to implement the same behavior through inheritance in other programming languages.
No. Providers blur the line between services (business logic) and brokers (integration layer). Brokers target particular disposable components within the system, but providers include more than just that.
ValueTask is a struct that is used to represent a task that returns a value. It is a value-based representation of a task that can be used to avoid allocations in the case where a task completes synchronously.
In order to abstract at an operational level, all example implementations in this section utilize the ValueTask type to provide an asynchronization abstraction. This approach encourages a consistent handling of asynchronous operations across all method implementations, promoting a clean and efficient abstraction that aligns with the Standard principles. By using ValueTask, we reduce memory allocations in scenarios where the result is often available immediately, thereby enhancing performance while maintaining code clarity.
Warnings are thrown by the compiler to alert you of potential issues in your code. It is important to address these warnings to ensure that your code is clean and free of potential bugs. Engineers and developers can resolve warnings by following the suggestions provided by the compiler. This may involve making changes to the code, refactoring, or updating dependencies.
It is important to regularly review and address warnings in your code to maintain code quality and ensure that your application runs smoothly.
Here is one way to resolve warnings in your code: To suppress a warning given by the IDE or compiler, you can configure your .csproj file using the element. This approach allows you to disable specific warnings globally across your project.
For example, adding CS1998 to a in your .csproj file will suppress the warning CS1998, which indicates an asynchronous method lacking await operators. If you need to suppress multiple warnings, you can list them separated by commas. This method provides a clean and consistent way to manage warnings without scattering #pragma warning directives throughout your code.
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<Nullable>disable</Nullable>
<ImplicitUsings>disable</ImplicitUsings>
<WarningLevel>CS1998</WarningLevel>
</PropertyGroup>
...[*] Implementing Abstract Components (Brokers)